00010000:00010000:2.0:1789762655.782571:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1600/0xdf5a7ac59f173a5b lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.782589:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff9c37d3bee100) 00010000:00010000:2.0:1789762655.782592:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762655.782594:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:2.0:1789762655.782669:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:11404:x1876702067202304:12345-192.168.203.37@tcp:36:mkdir.0 Request processed in 19129us (19225us total) trans 4294967315 rc 0/0 00000100:00100000:2.0:1789762655.782677:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 76 00000100:00100000:2.0:1789762655.794700:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067203328 00000100:00100000:2.0:1789762655.794706:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067203328 00000100:00100000:2.0:1789762655.794740:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 81 00000100:00100000:2.0:1789762655.794750:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:11405:x1876702067203328:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:2.0:1789762655.794787:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80b000/0xdf5a7ac59f173abd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.794797:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80b000/0xdf5a7ac59f173abd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.794805:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80b000/0xdf5a7ac59f173abd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762655.794990:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000733caf27 reply state 000000006d9484d3 transno 4294967317 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80b000/0xdf5a7ac59f173abd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.795003:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80b000/0xdf5a7ac59f173abd lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x4:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.795011:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x4:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762655.795014:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:2.0:1789762655.795017:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762655.795022:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x4:0x0]: returning size 16384 00000100:00100000:2.0:1789762655.795076:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:11405:x1876702067203328:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 327us (417us total) trans 4294967317 rc 0/0 00000100:00100000:2.0:1789762655.795085:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 81 00000100:00100000:2.0:1789762655.795667:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067203584 00000100:00100000:2.0:1789762655.795673:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067203584 00000100:00100000:2.0:1789762655.795710:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 16 00000100:00100000:2.0:1789762655.795719:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9a02c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:11405:x1876702067203584:12345-192.168.203.37@tcp:35:lfs.0 00000100:00100000:2.0:1789762655.795848:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9a02c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:11405:x1876702067203584:12345-192.168.203.37@tcp:35:lfs.0 Request processed in 129us (220us total) trans 4294967318 rc 0/0 00000100:00100000:2.0:1789762655.795856:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 16 00000100:00100000:2.0:1789762655.800257:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067203712 00000100:00100000:2.0:1789762655.800263:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067203712 00000100:00100000:2.0:1789762655.800309:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 82 00000100:00100000:2.0:1789762655.800319:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6baa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:11406:x1876702067203712:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:2.0:1789762655.800333:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762655.800345:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80b200/0xdf5a7ac59f173ac4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c34d expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800386:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ac00/0xdf5a7ac59f173acb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800398:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ac00/0xdf5a7ac59f173acb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800408:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ac00/0xdf5a7ac59f173acb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800420:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ba00/0xdf5a7ac59f173ad2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800435:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ba00/0xdf5a7ac59f173ad2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800443:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ba00/0xdf5a7ac59f173ad2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800511:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80bc00/0xdf5a7ac59f173ad9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800520:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80bc00/0xdf5a7ac59f173ad9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800529:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80bc00/0xdf5a7ac59f173ad9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762655.800548:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x4:0x0]: returning size 16384 00000004:00200000:2.0:1789762655.800552:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762655.800565:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80bc00/0xdf5a7ac59f173ad9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800576:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ac00/0xdf5a7ac59f173acb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800582:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ac00/0xdf5a7ac59f173acb lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800590:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ac00/0xdf5a7ac59f173acb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800596:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80ac00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762655.800601:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ac00/0xdf5a7ac59f173acb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800616:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ac00/0xdf5a7ac59f173acb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800629:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762655.800632:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762655.800636:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ac00/0xdf5a7ac59f173acb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800643:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ac00/0xdf5a7ac59f173acb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800658:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ba00/0xdf5a7ac59f173ad2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800665:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ba00/0xdf5a7ac59f173ad2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800672:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ba00/0xdf5a7ac59f173ad2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800679:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80ba00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762655.800683:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ba00/0xdf5a7ac59f173ad2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800690:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ba00/0xdf5a7ac59f173ad2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800701:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff9c37d3bee100) 00010000:00010000:2.0:1789762655.800704:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762655.800708:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ba00/0xdf5a7ac59f173ad2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800724:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80ba00/0xdf5a7ac59f173ad2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800756:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80b200/0xdf5a7ac59f173ac4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c34d expref: 14 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800773:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff80bc00/0xdf5a7ac59f173ad9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c34d expref: 13 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.800781:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x4:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762655.800784:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762655.800788:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff80bc00, rc 0) 00000100:00100000:2.0:1789762655.800832:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6baa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:11406:x1876702067203712:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 514us (619us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762655.800840:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 82 00000100:00100000:2.0:1789762655.813990:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067204864 00000100:00100000:2.0:1789762655.813997:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067204864 00000100:00100000:2.0:1789762655.814054:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 51 00000100:00100000:2.0:1789762655.814068:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067204864:12345-192.168.203.37@tcp:2:touch.0 00000100:00100000:2.0:1789762655.814418:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067204864:12345-192.168.203.37@tcp:2:touch.0 Request processed in 353us (455us total) trans 4294967323 rc 0/0 00000100:00100000:2.0:1789762655.814428:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 51 00000100:00100000:2.0:1789762655.862033:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067206400 00000100:00100000:2.0:1789762655.862039:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067206400 00000100:00100000:2.0:1789762655.862077:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 54 00000100:00100000:2.0:1789762655.862088:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:11414:x1876702067206400:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:2.0:1789762655.862105:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762655.862117:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b27736ed/0xdf5a7ac59f173b88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c37e expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.862142:0:9038:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f173b50 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c370 expref: 11 pid: 8054 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.862153:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762655.862158:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762655.862163:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b27736ed/0xdf5a7ac59f173b88 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000080000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c37e expref: 11 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.862193:0:9038:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f173b50 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c370 expref: 11 pid: 8054 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.862208:0:9038:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f173b50 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c370 expref: 11 pid: 8054 timeout: 229 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762655.862223:0:9038:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee8c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:9038:1876702074576000:192.168.203.37@tcp:104: 00000100:00100000:2.0:1789762655.862295:0:9038:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252ff00 going to sleep for 11 seconds 00000100:00100000:2.0:1789762655.863083:0:9038:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee8c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:9038:1876702074576000:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762655.863106:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b27736ed/0xdf5a7ac59f173b88 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c37e expref: 11 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762655.863119:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff80b200, rc 0) 00000100:00100000:2.0:1789762655.863156:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:11414:x1876702067206400:12345-192.168.203.37@tcp:101:dd.500 Request processed in 1071us (1183us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762655.863163:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 54 00000100:00100000:2.0:1789762655.865046:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067206656 00000100:00100000:2.0:1789762655.865052:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067206656 00002000:00010000:2.0:1789762655.865075:0:8057:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/3 (0->18446744073709551615) 00002000:00100000:2.0:1789762655.865080:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184f480 with x1876702067206656 ext(0->18446744073709551615) 00002000:00010000:2.0:1789762655.865088:0:8057:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184f480. 00000100:00100000:2.0:1789762655.865100:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 128582229199 time 128582340164 state 0x0 ref 1 00000100:00100000:2.0:1789762655.865112:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 128782340164 time 128582340164 state 0x1 ref 1 00000100:00100000:2.0:1789762655.865137:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 1 deadline 128782340164 time 129112216184 state 0x0 ref 1 00000100:00100000:2.0:1789762655.865152:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2398:x1876702067206656:12345-192.168.203.37@tcp:10:dd.500 00002000:00010000:2.0:1789762655.865354:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1789762655.865358:0:8057:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789762655 -> 1789762655 00002000:00010000:2.0:1789762655.865361:0:8057:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb atime from disk: 1789762655 -> 1789762655 00002000:00010000:2.0:1789762655.865364:0:8057:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789762655 -> 1789762655 00002000:00010000:2.0:1789762655.865366:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1789762655.865403:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:2398:x1876702067206656:12345-192.168.203.37@tcp:10:dd.500 Request processed in 255us (414us total) trans 4294967325 rc 0/0 00000100:00100000:2.0:1789762655.865414:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 34 00000100:00100000:2.0:1789762655.865420:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 1 deadline 128782340164 time 129112216184 state 0x2 ref 0 00002000:00010000:2.0:1789762655.865427:0:8057:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/3 (0->18446744073709551615) 00002000:00100000:2.0:1789762655.865431:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184f480 with x1876702067206656 ext(0->18446744073709551615) 00002000:00010000:2.0:1789762655.865444:0:8057:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184f480. 00000001:00080000:2.0:1789762655.886233:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967326 is committed 00000100:00100000:2.0:1789762656.068386:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 129315375035 time 129315456706 state 0x0 ref 1 00000100:00100000:2.0:1789762656.068435:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067207040:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762656.068492:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 1048576, mtime: 0x6aad9c5f, atime: 0x6aad9c5f, ctime: 0x6aad9c5f, blocks: 0x803 00010000:00010000:2.0:1789762656.068503:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cae9fdff/0xdf5a7ac59f173b96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.068520:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=129s v=(0 0) 00010000:00010000:2.0:1789762656.068525:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762656.068531:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cae9fdff/0xdf5a7ac59f173b96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.068546:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cae9fdff/0xdf5a7ac59f173b96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762656.080968:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967327 is committed 00002000:00010000:2.0:1789762656.288225:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:2.0:1789762656.288234:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:2.0:1789762656.288247:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f173b9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.288262:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f173b9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.288272:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f173b9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.288288:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f0a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762656.288293:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f173b9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.288302:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f173b9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.288321:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f173b9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.288328:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f173b9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762656.288414:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9af100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067207168:12345-192.168.203.37@tcp:4:dd.500 Request processed in 19846us (204530us total) trans 4294967328 rc 0/0 00000100:00100000:2.0:1789762656.288425:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 37 00000100:00100000:2.0:1789762656.288433:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 129515456706 time 129515591529 state 0x2 ref 0 00000100:00100000:2.0:1789762656.292438:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067207296 00000100:00100000:2.0:1789762656.292449:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067207296 00000100:00100000:2.0:1789762656.292494:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 129515456706 time 129515591529 state 0x0 ref 1 00000100:00100000:2.0:1789762656.292506:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 129715591529 time 129515591529 state 0x1 ref 1 00000100:00100000:2.0:1789762656.668849:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 129915810167 time 129915918473 state 0x0 ref 1 00000100:00100000:2.0:1789762656.668885:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067207424:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762656.669007:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 4194304, mtime: 0x6aad9c60, atime: 0x6aad9c5f, ctime: 0x6aad9c60, blocks: 0x2007 00010000:00010000:2.0:1789762656.669017:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.669034:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=129s v=(0 0) 00010000:00010000:2.0:1789762656.669040:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762656.669048:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.669063:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762656.681796:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:2.0:1789762656.681803:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:2.0:1789762656.681817:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.681832:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.681842:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.681851:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80ac00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762656.681856:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.681879:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.681899:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762656.681909:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762656.682019:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067207424:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13147us (188491us total) trans 4294967330 rc 0/0 00000100:00100000:2.0:1789762656.682031:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 39 00000100:00100000:2.0:1789762656.682039:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 129915810167 time 129915918473 state 0x2 ref 0 00000001:00080000:2.0:1789762656.884567:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967331 is committed 00000100:00100000:2.0:1789762656.888634:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067207680 00000100:00100000:2.0:1789762656.888642:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067207680 00000100:00100000:2.0:1789762656.888684:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 130115918473 time 130115993527 state 0x0 ref 1 00000100:00100000:2.0:1789762656.888700:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 130315993527 time 130115993527 state 0x1 ref 1 00000001:00080000:2.0:1789762657.083166:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967332 is committed 00000100:00100000:2.0:1789762657.269143:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 130516130221 time 130516213980 state 0x0 ref 1 00000100:00100000:2.0:1789762657.269172:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067207808:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762657.269250:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 7340032, mtime: 0x6aad9c60, atime: 0x6aad9c5f, ctime: 0x6aad9c60, blocks: 0x3807 00010000:00010000:2.0:1789762657.269291:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bc0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762657.269319:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=130s v=(0 0) 00010000:00010000:2.0:1789762657.269328:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762657.269340:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762657.269359:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762657.292057:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:2.0:1789762657.292069:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:2.0:1789762657.292088:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762657.292108:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bc0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762657.292121:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762657.292131:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80ac00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762657.292137:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762657.292151:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bc0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762657.292195:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bc0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762657.292207:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173bc0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762657.292342:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2398:x1876702067207808:12345-192.168.203.37@tcp:4:dd.500 Request processed in 23172us (206345us total) trans 4294967333 rc 0/0 00000100:00100000:2.0:1789762657.292359:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 42 00000100:00100000:2.0:1789762657.292371:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 130516130221 time 130516213980 state 0x2 ref 0 00000001:00080000:2.0:1789762657.292431:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967333 is committed 00000001:00080000:2.0:1789762657.322973:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967317 is committed 00000001:00080000:2.0:1789762657.323010:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967320 is committed 00000001:00080000:2.0:1789762657.491157:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967334 is committed 00000100:00100000:2.0:1789762657.686164:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067208192 00000100:00100000:2.0:1789762657.686173:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067208192 00000100:00100000:2.0:1789762657.686210:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 130916368753 time 130916496869 state 0x0 ref 1 00000100:00100000:2.0:1789762657.686219:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 131116496869 time 130916496869 state 0x1 ref 1 00000100:00100000:2.0:1789762657.884848:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067208320 00000100:00100000:2.0:1789762657.884858:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067208320 00000100:00100000:2.0:1789762657.884900:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 131116496869 time 131116640127 state 0x0 ref 1 00000100:00100000:2.0:1789762657.884939:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 131316640127 time 131116640127 state 0x1 ref 1 00000100:00100000:2.0:1789762658.069632:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 131316640127 time 131316702937 state 0x0 ref 1 00000100:00100000:2.0:1789762658.069706:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067208320:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762658.069779:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 11534336, mtime: 0x6aad9c61, atime: 0x6aad9c5f, ctime: 0x6aad9c61, blocks: 0x5807 00010000:00010000:2.0:1789762658.069787:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008f9823b2/0xdf5a7ac59f173bdc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762658.069801:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=131s v=(0 0) 00010000:00010000:2.0:1789762658.069804:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762658.069809:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008f9823b2/0xdf5a7ac59f173bdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762658.069820:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008f9823b2/0xdf5a7ac59f173bdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762658.269705:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 131516702937 time 131516777410 state 0x0 ref 1 00000100:00100000:2.0:1789762658.269731:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a35880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067208448:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762658.269778:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 12582912, mtime: 0x6aad9c61, atime: 0x6aad9c5f, ctime: 0x6aad9c61, blocks: 0x6007 00010000:00010000:2.0:1789762658.269785:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ac541fa/0xdf5a7ac59f173be3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762658.269798:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=131s v=(0 0) 00010000:00010000:2.0:1789762658.269802:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762658.269806:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ac541fa/0xdf5a7ac59f173be3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762658.269835:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ac541fa/0xdf5a7ac59f173be3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762658.681841:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967340 is committed 00000100:00100000:2.0:1789762658.684509:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067208832 00000100:00100000:2.0:1789762658.684520:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067208832 00000100:00100000:2.0:1789762658.684557:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 131916859453 time 131916956675 state 0x0 ref 1 00000100:00100000:2.0:1789762658.684567:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 132116956675 time 131916956675 state 0x1 ref 1 00000100:00100000:2.0:1789762658.760116:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067209472 00000100:00100000:2.0:1789762658.760124:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067209472 00000100:00100000:2.0:1789762658.760185:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 79 00000100:00100000:2.0:1789762658.760205:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:2397:x1876702067209216:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762658.760277:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:2397:x1876702067209216:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 74us (284us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762658.760318:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 79 00000100:00100000:2.0:1789762658.760346:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 81 00000100:00100000:2.0:1789762658.760357:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2397:x1876702067209088:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762658.760391:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2397:x1876702067209088:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 35us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762658.760400:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 81 00000100:00080000:2.0:1789762658.764252:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762658.764258:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1789762658.764332:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1789762658.881997:0:36:0:(genops.c:844:class_export_destroy()) destroying export ffff9c37ff6d1800/2a924629-ba65-41c7-b7b8-60afcac2e2ea for lustre-OST0000 00000020:00000080:2.0:1789762658.882057:0:36:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9c37ff6d3800 for lustre-OST0000 00000100:00100000:2.0:1789762658.884488:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067209984 00000100:00100000:2.0:1789762658.884498:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067209984 00000100:00100000:2.0:1789762658.884537:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 132116956675 time 132117059282 state 0x0 ref 1 00000100:00100000:2.0:1789762658.884547:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 132317059282 time 132117059282 state 0x1 ref 1 00000100:00100000:2.0:1789762659.070125:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 132317059282 time 132317190556 state 0x0 ref 1 00000100:00100000:2.0:1789762659.070172:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067209984:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762659.070264:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 16777216, mtime: 0x6aad9c62, atime: 0x6aad9c5f, ctime: 0x6aad9c62, blocks: 0x8007 00010000:00010000:2.0:1789762659.070273:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173bff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.070288:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=132s v=(0 0) 00010000:00010000:2.0:1789762659.070292:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762659.070298:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173bff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.070311:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173bff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762659.270240:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 132517190556 time 132517293502 state 0x0 ref 1 00000100:00100000:2.0:1789762659.270280:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067210112:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762659.270357:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17825792, mtime: 0x6aad9c62, atime: 0x6aad9c5f, ctime: 0x6aad9c62, blocks: 0x8807 00010000:00010000:2.0:1789762659.270382:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173c06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.270398:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=132s v=(0 0) 00010000:00010000:2.0:1789762659.270403:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762659.270409:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173c06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.270423:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173c06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762659.288192:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967343 is committed 00000001:00080000:2.0:1789762659.494490:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967344 is committed 00000001:00080000:2.0:1789762659.683187:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967345 is committed 00000100:00100000:2.0:1789762659.685907:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067210496 00000100:00100000:2.0:1789762659.685915:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067210496 00000100:00100000:2.0:1789762659.685969:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 132917408553 time 132917610297 state 0x0 ref 1 00000100:00100000:2.0:1789762659.685983:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 133117610297 time 132917610297 state 0x1 ref 1 00000100:00100000:2.0:1789762659.840354:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee8c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074576896:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.840412:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762659.840449:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee8c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074577408:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.840484:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762659.840513:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfec700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074577920:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.840541:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762659.840583:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfedc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074578432:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.840645:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762659.840687:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:2.0:1789762659.840701:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee8c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074577024:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.840738:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762659.840771:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee8c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074577536:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.840810:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762659.840844:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074578048:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.840880:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762659.840973:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762659.840983:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb930380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074578560:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762659.841030:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762659.841123:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074576768 00000100:00100000:2.0:1789762659.841130:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074576768 00000100:00100000:2.0:1789762659.841183:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 278 00000100:00100000:2.0:1789762659.841204:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1579c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074576768:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.841259:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1579c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074576768:12345-0@lo:400:kworker.0 Request processed in 61us (966us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762659.841278:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 278 00000100:00100000:2.0:1789762659.841334:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074577920 00000100:00100000:2.0:1789762659.841342:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074577920 00000100:00100000:2.0:1789762659.841390:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074578432 00000100:00100000:2.0:1789762659.841397:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074578432 00000100:00100000:2.0:1789762659.841438:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074578176 00000100:00100000:2.0:1789762659.841443:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074578176 00000100:00100000:2.0:1789762659.841460:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074578048 00000100:00100000:2.0:1789762659.841466:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074578048 00000100:00100000:2.0:1789762659.841491:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 34 00000100:00100000:2.0:1789762659.841505:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074577920:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.841573:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074577920:12345-0@lo:400:kworker.0 Request processed in 72us (1030us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762659.841585:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 34 00000100:00100000:2.0:1789762659.841613:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 35 00000100:00100000:2.0:1789762659.841625:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074578432:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.841665:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074578432:12345-0@lo:400:kworker.0 Request processed in 41us (1020us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762659.841676:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 35 00000100:00100000:2.0:1789762659.841703:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 37 00000100:00100000:2.0:1789762659.841718:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074578048:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.841767:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074578048:12345-0@lo:400:kworker.0 Request processed in 55us (889us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762659.841776:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 37 00000100:00100000:2.0:1789762659.841810:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 87 00000100:00100000:2.0:1789762659.841822:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074577536:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.841856:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074577536:12345-0@lo:400:kworker.0 Request processed in 37us (1046us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762659.841862:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 87 00000100:00080000:2.0:1789762659.841888:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762659.841908:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762659.841911:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762659.841914:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762659.841964:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c94a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074578688:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762659.841996:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762659.842047:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee760380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074576640:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762659.842078:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee8c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074577152:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.842091:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee8c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074577664:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.842102:0:3424:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967301 gen 2 00000004:00080000:2.0:1789762659.842109:0:3424:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37ff684380, transno 4294967301 00000100:00100000:2.0:1789762659.842122:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfeea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074578176:0@lo:400:kworker.0 00000001:00080000:2.0:1789762659.855137:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967326 is committed 00000020:00080000:2.0:1789762659.855147:0:7221:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9c37f5fae780 refcount 2 committed 0 result 0 batchid 2 00000020:00080000:2.0:1789762659.855152:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37ff763200 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9c37f5fae840 00000020:00080000:2.0:1789762659.855155:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x2:0x0].2 00000020:00080000:2.0:1789762659.855161:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37ff763e00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9c36cb90b400 00000020:00080000:2.0:1789762659.855168:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x2:0x0].2 00000004:00080000:2.0:1789762659.855186:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 2 committed 00000004:00080000:2.0:1789762659.855199:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 2 committed 00000020:00080000:2.0:1789762659.855226:0:7221:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 1 batchid updated 00000100:00100000:2.0:1789762659.855380:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c94a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074578944:0@lo:6:osp-syn-1-1.0 00000100:00100000:2.0:1789762659.855454:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762659.855715:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074579200 00000100:00100000:2.0:1789762659.855721:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074579200 00000100:00100000:2.0:1789762659.855769:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 85 00000100:00100000:2.0:1789762659.855785:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:7224:x1876702074579200:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762659.856263:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074579200:12345-0@lo:1000:osp_up0-1.0 Request processed in 480us (582us total) trans 4294967327 rc 0/0 00000100:00100000:2.0:1789762659.856292:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 85 00010000:00010000:2.0:1789762659.856481:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762659.856490:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f173c22 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.856509:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f0c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762659.856514:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f173c22 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.856524:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f173c22 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.856540:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f173c22 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.856547:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f173c22 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.856570:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762659.856572:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762659.856576:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1789762659.856674:0:3426:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37c94a1180/2, rc 0, transno 4294967321 00000100:00100000:2.0:1789762659.856697:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c94a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074578944:0@lo:6:osp-syn-1-1.0 00002000:00010000:2.0:1789762659.883997:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:2.0:1789762659.884004:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:2.0:1789762659.884017:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ac3605a/0xdf5a7ac59f173c29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.884027:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ac3605a/0xdf5a7ac59f173c29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.884046:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ac3605a/0xdf5a7ac59f173c29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.884051:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36cb8e2a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762659.884055:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ac3605a/0xdf5a7ac59f173c29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.884062:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ac3605a/0xdf5a7ac59f173c29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.884077:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ac3605a/0xdf5a7ac59f173c29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.884083:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ac3605a/0xdf5a7ac59f173c29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762659.884158:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067210496:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13492us (198298us total) trans 4294967347 rc 0/0 00000100:00100000:2.0:1789762659.884166:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 55 00000100:00100000:2.0:1789762659.884174:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 133117610297 time 133117718387 state 0x2 ref 0 00000100:00100000:2.0:1789762660.070752:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 133317718387 time 133317823548 state 0x0 ref 1 00000100:00100000:2.0:1789762660.070793:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067210624:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762660.070874:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22020096, mtime: 0x6aad9c63, atime: 0x6aad9c5f, ctime: 0x6aad9c63, blocks: 0xa807 00010000:00010000:2.0:1789762660.070911:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ac541fa/0xdf5a7ac59f173c30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.070985:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=133s v=(0 0) 00010000:00010000:2.0:1789762660.070992:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762660.071000:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ac541fa/0xdf5a7ac59f173c30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.071020:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ac541fa/0xdf5a7ac59f173c30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762660.497870:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067211008 00000100:00100000:2.0:1789762660.497955:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067211008 00000100:00100000:2.0:1789762660.498025:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 133717902852 time 133718058213 state 0x0 ref 1 00000100:00100000:2.0:1789762660.498041:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 133918058213 time 133718058213 state 0x1 ref 1 00000001:00080000:2.0:1789762660.697267:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967351 is committed 00000100:00100000:2.0:1789762660.701977:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067211136 00000100:00100000:2.0:1789762660.701991:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067211136 00000100:00100000:2.0:1789762660.702048:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 133918058213 time 133918279071 state 0x0 ref 1 00000100:00100000:2.0:1789762660.702066:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 134118279071 time 133918279071 state 0x1 ref 1 00000001:00080000:2.0:1789762660.874095:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967326 is committed 00000001:00080000:2.0:1789762660.882738:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967352 is committed 00000100:00100000:2.0:1789762660.885282:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762661.071337:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 134318330773 time 134318410762 state 0x0 ref 1 00000100:00100000:2.0:1789762661.071362:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067211264:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762661.071418:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 27262976, mtime: 0x6aad9c64, atime: 0x6aad9c5f, ctime: 0x6aad9c64, blocks: 0xd007 00010000:00010000:2.0:1789762661.071426:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cae9fdff/0xdf5a7ac59f173c53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762661.071439:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=134s v=(0 0) 00010000:00010000:2.0:1789762661.071443:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762661.071447:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cae9fdff/0xdf5a7ac59f173c53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762661.071458:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cae9fdff/0xdf5a7ac59f173c53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762661.085146:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762661.285644:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:2.0:1789762661.482968:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967355 is committed 00000001:00080000:2.0:1789762661.682363:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967356 is committed 00000001:00080000:2.0:1789762661.882852:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967357 is committed 00000100:00100000:2.0:1789762661.885318:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:2.0:1789762662.082268:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967358 is committed 00000001:00080000:2.0:1789762662.284186:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967359 is committed 00000100:00100000:2.0:1789762662.286706:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:2.0:1789762662.411864:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967321 is committed 00000001:00100000:2.0:1789762662.411891:0:9000:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 2a924629-ba65-41c7-b7b8-60afcac2e2ea 00000020:00000080:2.0:1789762662.411894:0:9000:0:(genops.c:885:class_export_put()) final put ffff9c37f7b6a800/2a924629-ba65-41c7-b7b8-60afcac2e2ea 00000001:00080000:2.0:1789762662.483323:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967360 is committed 00000100:00100000:2.0:1789762662.485626:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:2.0:1789762662.684173:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967361 is committed 00000100:00100000:2.0:1789762662.686553:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:2.0:1789762662.883090:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967362 is committed 00000100:00100000:2.0:1789762662.885509:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:2.0:1789762663.040034:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789762663.084096:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:2.0:1789762663.084103:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 73735 -> 75783 00010000:00010000:2.0:1789762663.084116:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f173c99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.084128:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f173c99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.084162:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f173c99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.084175:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762663.084185:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f173c99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.084200:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f173c99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.084229:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f173c99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.084245:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f173c99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762663.084391:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067212544:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12241us (198878us total) trans 4294967363 rc 0/0 00000100:00100000:2.0:1789762663.084401:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 71 00000100:00100000:2.0:1789762663.084409:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 136319111610 time 136319202718 state 0x2 ref 0 00000100:00100000:2.0:1789762663.087152:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762663.286155:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762663.687713:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067213056 00000100:00100000:2.0:1789762663.687722:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067213056 00000100:00100000:2.0:1789762663.687766:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 136919334275 time 136919419925 state 0x0 ref 1 00000100:00100000:2.0:1789762663.687779:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 137119419925 time 136919419925 state 0x1 ref 1 00000100:00100000:2.0:1789762663.879964:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067213440 00000100:00100000:2.0:1789762663.879970:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067213440 00000100:00100000:2.0:1789762663.879994:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067213696 00000100:00100000:2.0:1789762663.880000:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067213696 00000100:00100000:2.0:1789762663.880053:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 92 00000100:00100000:2.0:1789762663.880070:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067213696:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762663.880116:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067213696:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 49us (194us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762663.880130:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 92 00000100:00100000:2.0:1789762663.880162:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067213568 00000100:00100000:2.0:1789762663.880165:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067213568 00000100:00100000:2.0:1789762663.880200:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 38 00000100:00100000:2.0:1789762663.880207:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067213568:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762663.880238:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067213568:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 32us (484us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762663.880243:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 38 00000100:00100000:2.0:1789762664.072553:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 137319543145 time 137319626868 state 0x0 ref 1 00000100:00100000:2.0:1789762664.072599:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff685880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067213952:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762664.072648:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 42991616, mtime: 0x6aad9c67, atime: 0x6aad9c5f, ctime: 0x6aad9c67, blocks: 0x14807 00010000:00010000:2.0:1789762664.072657:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b27736ed/0xdf5a7ac59f173cbc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.072670:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=137s v=(0 0) 00010000:00010000:2.0:1789762664.072674:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762664.072678:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b27736ed/0xdf5a7ac59f173cbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.072688:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b27736ed/0xdf5a7ac59f173cbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762664.687743:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762664.960340:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb931880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074579712:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762664.960386:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762664.960417:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee761180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074580096:0@lo:400:kworker.0 00000100:00100000:2.0:1789762664.960462:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762664.960507:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ebaa3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074580608:0@lo:400:kworker.0 00000100:00100000:2.0:1789762664.960551:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762664.960594:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ebaa0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074581120:0@lo:400:kworker.0 00000100:00100000:2.0:1789762664.960640:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762664.960692:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074580096 00000100:00100000:2.0:1789762664.960698:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074580096 00000100:00100000:2.0:1789762664.960752:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:2.0:1789762664.960771:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d157ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074580096:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762664.960827:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d157ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074580096:12345-0@lo:400:kworker.0 Request processed in 62us (366us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762664.960838:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:2.0:1789762664.960958:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 95 00000100:00100000:2.0:1789762664.960972:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb933800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074580736:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762664.961032:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb933800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074580736:12345-0@lo:400:kworker.0 Request processed in 63us (608us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762664.961050:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 95 00000100:00100000:2.0:1789762664.961117:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee760380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074580352:0@lo:400:kworker.0 00000100:00100000:2.0:1789762664.961142:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ebaa2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074580864:0@lo:400:kworker.0 00000100:00100000:2.0:1789762664.961172:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f60a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074579840:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762664.961233:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ebaa0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074581120:0@lo:400:kworker.0 00000100:00100000:2.0:1789762664.961902:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 98 00000004:00080000:2.0:1789762664.968051:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 4 committed 00000004:00080000:2.0:1789762664.968068:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 3 committed 00000004:00080000:2.0:1789762664.968077:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 4 committed 00000020:00080000:2.0:1789762664.968086:0:7221:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 2 batchid updated 00000004:00080000:2.0:1789762664.968099:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 3 committed 00000100:00100000:2.0:1789762664.968140:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f60a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074581760:0@lo:6:osp-syn-1-1.0 00000100:00100000:2.0:1789762664.968186:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789762664.968446:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762664.968459:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000024f15b62/0xdf5a7ac59f173ce6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.968474:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762664.968481:0:6408:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000024f15b62/0xdf5a7ac59f173ce6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.968493:0:6408:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000024f15b62/0xdf5a7ac59f173ce6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.968513:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000024f15b62/0xdf5a7ac59f173ce6 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.968524:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024f15b62/0xdf5a7ac59f173ce6 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.968552:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762664.968555:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762664.968559:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762664.968702:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074582272 00000100:00100000:2.0:1789762664.968707:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074582272 00000100:00100000:2.0:1789762664.968740:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 90 00000100:00100000:2.0:1789762664.968748:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:7224:x1876702074582272:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762664.969156:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074582272:12345-0@lo:1000:osp_up0-1.0 Request processed in 408us (480us total) trans 4294967328 rc 0/0 00000100:00100000:2.0:1789762664.969173:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 90 00000100:00100000:2.0:1789762664.969814:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762664.969952:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:2.0:1789762665.088987:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762665.287185:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762665.486789:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:2.0:1789762665.991422:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967328 is committed 00002000:00010000:2.0:1789762666.084280:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 53477376 -> 54525952 00002000:00010000:2.0:1789762666.084286:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 104457 -> 106505 00010000:00010000:2.0:1789762666.084298:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f173d10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.084312:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f173d10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.084322:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f173d10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.084331:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762666.084336:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f173d10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.084347:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f173d10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.084364:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f173d10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.084386:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f173d10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762666.084453:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067215488:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10978us (194388us total) trans 4294967379 rc 0/0 00000100:00100000:2.0:1789762666.084463:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 86 00000100:00100000:2.0:1789762666.084470:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 139320436260 time 139320529457 state 0x2 ref 0 00002000:00010000:2.0:1789762666.285110:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 54525952 -> 55574528 00002000:00010000:2.0:1789762666.285116:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 106505 -> 108553 00010000:00010000:2.0:1789762666.285126:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f173d17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.285134:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f173d17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.285141:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f173d17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.285146:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762666.285149:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f173d17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.285156:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f173d17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.285181:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f173d17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.285187:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f173d17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762666.285246:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067215616:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11677us (198461us total) trans 4294967380 rc 0/0 00000100:00100000:2.0:1789762666.285253:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 87 00000100:00100000:2.0:1789762666.285259:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 139520529457 time 139520625190 state 0x2 ref 0 00002000:00010000:2.0:1789762666.486257:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 55574528 -> 56623104 00002000:00010000:2.0:1789762666.486263:0:8059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789762665 -> 1789762666 00002000:00010000:2.0:1789762666.486266:0:8059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789762665 -> 1789762666 00002000:00010000:2.0:1789762666.486269:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 108553 -> 110601 00010000:00010000:2.0:1789762666.486282:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006989fd2a/0xdf5a7ac59f173d1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.486295:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006989fd2a/0xdf5a7ac59f173d1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.486305:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006989fd2a/0xdf5a7ac59f173d1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.486311:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36cb8e3400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762666.486321:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006989fd2a/0xdf5a7ac59f173d1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.486328:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006989fd2a/0xdf5a7ac59f173d1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.486342:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006989fd2a/0xdf5a7ac59f173d1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.486347:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006989fd2a/0xdf5a7ac59f173d1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762666.486417:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067215744:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12733us (198949us total) trans 4294967381 rc 0/0 00000100:00100000:2.0:1789762666.486426:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 88 00000100:00100000:2.0:1789762666.486432:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 139720625190 time 139720719608 state 0x2 ref 0 00002000:00010000:2.0:1789762666.685143:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 56623104 -> 57671680 00002000:00010000:2.0:1789762666.685150:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 110601 -> 112649 00010000:00010000:2.0:1789762666.685161:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f173d25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.685173:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f173d25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.685183:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f173d25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.685202:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762666.685206:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f173d25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.685213:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f173d25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.685226:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f173d25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.685232:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f173d25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762666.685287:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067215872:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11492us (196497us total) trans 4294967382 rc 0/0 00000100:00100000:2.0:1789762666.685294:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 89 00000100:00100000:2.0:1789762666.685300:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 139920719608 time 139920850559 state 0x2 ref 0 00000100:00100000:2.0:1789762666.687567:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067216000 00000100:00100000:2.0:1789762666.687576:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067216000 00000100:00100000:2.0:1789762666.687612:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 139920719608 time 139920850559 state 0x0 ref 1 00000100:00100000:2.0:1789762666.687621:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140120850559 time 139920850559 state 0x1 ref 1 00002000:00010000:2.0:1789762666.887981:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 57671680 -> 58720256 00002000:00010000:2.0:1789762666.887990:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 112649 -> 114697 00010000:00010000:2.0:1789762666.888016:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f173d2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.888031:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f173d2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.888044:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f173d2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.888054:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f1600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762666.888061:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f173d2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.888074:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f173d2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.888098:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f173d2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762666.888109:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f173d2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762666.888198:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067216000:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14276us (200664us total) trans 4294967383 rc 0/0 00000100:00100000:2.0:1789762666.888213:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 90 00000100:00100000:2.0:1789762666.888222:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140120850559 time 140120969047 state 0x2 ref 0 00000100:00100000:2.0:1789762666.891203:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067216128 00000100:00100000:2.0:1789762666.891221:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067216128 00000100:00100000:2.0:1789762666.891268:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140120850559 time 140120969047 state 0x0 ref 1 00000100:00100000:2.0:1789762666.891284:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140320969047 time 140120969047 state 0x1 ref 1 00000100:00100000:2.0:1789762667.008198:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074582400 00000100:00100000:2.0:1789762667.008208:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074582400 00000100:00100000:2.0:1789762667.008266:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 91 00000100:00100000:2.0:1789762667.008278:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ebaa0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074582400:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762667.008316:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762667.008373:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074582400:12345-0@lo:41:osp-pre-1-0.0 Request processed in 97us (234us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762667.008382:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 91 00000100:00100000:2.0:1789762667.073969:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140320969047 time 140321040804 state 0x0 ref 1 00000100:00100000:2.0:1789762667.073996:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067216128:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762667.074047:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 58720256, mtime: 0x6aad9c6a, atime: 0x6aad9c5f, ctime: 0x6aad9c6a, blocks: 0x1c009 00010000:00010000:2.0:1789762667.074057:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.074074:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=140s v=(0 0) 00010000:00010000:2.0:1789762667.074079:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762667.074087:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.074102:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762667.085941:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 58720256 -> 59768832 00002000:00010000:2.0:1789762667.085946:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 114697 -> 116745 00010000:00010000:2.0:1789762667.085957:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.085966:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.085972:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.085977:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80b800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762667.085981:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.085988:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.086002:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.086006:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762667.086068:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067216128:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12082us (194918us total) trans 4294967384 rc 0/0 00000100:00100000:2.0:1789762667.086082:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 91 00000100:00100000:2.0:1789762667.086088:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140320969047 time 140321040804 state 0x2 ref 0 00010000:00010000:2.0:1789762667.136032:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762667.274059:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140521040804 time 140521129754 state 0x0 ref 1 00000100:00100000:2.0:1789762667.274091:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ebaa1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067216256:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762667.274145:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 59768832, mtime: 0x6aad9c6a, atime: 0x6aad9c5f, ctime: 0x6aad9c6a, blocks: 0x1c809 00010000:00010000:2.0:1789762667.274155:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.274173:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=140s v=(0 0) 00010000:00010000:2.0:1789762667.274179:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762667.274187:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.274203:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762667.285408:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 59768832 -> 60817408 00002000:00010000:2.0:1789762667.285413:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 116745 -> 118793 00010000:00010000:2.0:1789762667.285422:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.285432:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.285449:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.285455:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80b800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762667.285458:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.285464:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.285478:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.285482:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762667.285546:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067216256:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11467us (196493us total) trans 4294967385 rc 0/0 00000100:00100000:2.0:1789762667.285554:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 92 00000100:00100000:2.0:1789762667.285560:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140521040804 time 140521129754 state 0x2 ref 0 00002000:00010000:2.0:1789762667.485744:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 60817408 -> 61865984 00002000:00010000:2.0:1789762667.485749:0:8059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789762666 -> 1789762667 00002000:00010000:2.0:1789762667.485751:0:8059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789762666 -> 1789762667 00002000:00010000:2.0:1789762667.485753:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 118793 -> 120841 00010000:00010000:2.0:1789762667.485769:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e07a80/0xdf5a7ac59f173d41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.485778:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e07a80/0xdf5a7ac59f173d41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.485785:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e07a80/0xdf5a7ac59f173d41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.485789:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36cb8e2400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762667.485793:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e07a80/0xdf5a7ac59f173d41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.485800:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e07a80/0xdf5a7ac59f173d41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.485813:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e07a80/0xdf5a7ac59f173d41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.485818:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e07a80/0xdf5a7ac59f173d41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762667.485892:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067216384:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11740us (197796us total) trans 4294967386 rc 0/0 00000100:00100000:2.0:1789762667.485901:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 93 00000100:00100000:2.0:1789762667.485907:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140721129754 time 140721189959 state 0x2 ref 0 00000100:00100000:2.0:1789762667.488447:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067216512 00000100:00100000:2.0:1789762667.488460:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067216512 00000100:00100000:2.0:1789762667.488493:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140721129754 time 140721189959 state 0x0 ref 1 00000100:00100000:2.0:1789762667.488503:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140921189959 time 140721189959 state 0x1 ref 1 00000001:00080000:2.0:1789762667.528587:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967322 is committed 00002000:00010000:2.0:1789762667.685743:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 61865984 -> 62914560 00002000:00010000:2.0:1789762667.685748:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 120841 -> 122889 00010000:00010000:2.0:1789762667.685757:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071438c9f/0xdf5a7ac59f173d48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.685765:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071438c9f/0xdf5a7ac59f173d48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.685771:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071438c9f/0xdf5a7ac59f173d48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.685777:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36cb8e2000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762667.685780:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071438c9f/0xdf5a7ac59f173d48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.685787:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071438c9f/0xdf5a7ac59f173d48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.685805:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071438c9f/0xdf5a7ac59f173d48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.685814:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071438c9f/0xdf5a7ac59f173d48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762667.685904:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067216512:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11657us (197498us total) trans 4294967387 rc 0/0 00000100:00100000:2.0:1789762667.685914:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 94 00000100:00100000:2.0:1789762667.685952:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140921189959 time 140921301802 state 0x2 ref 0 00000100:00100000:2.0:1789762667.688123:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067216640 00000100:00100000:2.0:1789762667.688131:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067216640 00000100:00100000:2.0:1789762667.688166:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 140921189959 time 140921301802 state 0x0 ref 1 00000100:00100000:2.0:1789762667.688174:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 141121301802 time 140921301802 state 0x1 ref 1 00000100:00100000:2.0:1789762667.874275:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 141121301802 time 141121347308 state 0x0 ref 1 00000100:00100000:2.0:1789762667.874300:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ebaa2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067216640:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762667.874359:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 62914560, mtime: 0x6aad9c6b, atime: 0x6aad9c5f, ctime: 0x6aad9c6b, blocks: 0x1e009 00010000:00010000:2.0:1789762667.874366:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.874379:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=141s v=(0 0) 00010000:00010000:2.0:1789762667.874383:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762667.874387:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.874400:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762667.886731:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 62914560 -> 63963136 00002000:00010000:2.0:1789762667.886736:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 122889 -> 124937 00010000:00010000:2.0:1789762667.886747:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.886757:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.886763:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.886768:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80b800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762667.886772:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.886779:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.886794:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762667.886802:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762667.886883:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067216640:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12591us (198795us total) trans 4294967388 rc 0/0 00000100:00100000:2.0:1789762667.886898:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 95 00000100:00100000:2.0:1789762667.886903:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 141121301802 time 141121347308 state 0x2 ref 0 00000100:00100000:2.0:1789762668.074405:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 141321347308 time 141321476602 state 0x0 ref 1 00000100:00100000:2.0:1789762668.074448:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ebaa0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067216768:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762668.074499:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 63963136, mtime: 0x6aad9c6b, atime: 0x6aad9c5f, ctime: 0x6aad9c6b, blocks: 0x1e809 00010000:00010000:2.0:1789762668.074508:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762668.074521:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=141s v=(0 0) 00010000:00010000:2.0:1789762668.074525:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762668.074529:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762668.074541:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762668.086855:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 63963136 -> 65011712 00002000:00010000:2.0:1789762668.086860:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 124937 -> 126985 00010000:00010000:2.0:1789762668.086872:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762668.086882:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762668.086899:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762668.086905:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80b800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762668.086909:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762668.086915:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762668.086948:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762668.086954:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762668.087016:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067216768:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12594us (197585us total) trans 4294967389 rc 0/0 00000100:00100000:2.0:1789762668.087023:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 96 00000100:00100000:2.0:1789762668.087030:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 141321347308 time 141321476602 state 0x2 ref 0 00000100:00100000:2.0:1789762668.293217:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067217024 00000100:00100000:2.0:1789762668.293225:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067217024 00000100:00100000:2.0:1789762668.293263:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 141521476602 time 141521533647 state 0x0 ref 1 00000100:00100000:2.0:1789762668.293272:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 141721533647 time 141521533647 state 0x1 ref 1 00000100:00100000:2.0:1789762668.488684:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067217152 00000100:00100000:2.0:1789762668.488692:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067217152 00000100:00100000:2.0:1789762668.488726:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 141721533647 time 141721661680 state 0x0 ref 1 00000100:00100000:2.0:1789762668.488735:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 141921661680 time 141721661680 state 0x1 ref 1 00000100:00100000:2.0:1789762668.674671:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 141921661680 time 141921743962 state 0x0 ref 1 00000100:00100000:2.0:1789762668.674701:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6096d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067217152:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762668.674752:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 67108864, mtime: 0x6aad9c6c, atime: 0x6aad9c5f, ctime: 0x6aad9c6c, blocks: 0x20009 00010000:00010000:2.0:1789762668.674759:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762668.674772:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=141s v=(0 0) 00010000:00010000:2.0:1789762668.674775:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762668.674780:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762668.674791:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000226b7ab/0xdf5a7ac59f173d6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762668.688968:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762668.874764:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 142121743962 time 142121836489 state 0x0 ref 1 00000100:00100000:2.0:1789762668.874794:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067217280:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762668.874858:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 68157440, mtime: 0x6aad9c6c, atime: 0x6aad9c5f, ctime: 0x6aad9c6c, blocks: 0x20809 00010000:00010000:2.0:1789762668.874879:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ac541fa/0xdf5a7ac59f173d72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762668.874892:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=142s v=(0 0) 00010000:00010000:2.0:1789762668.874896:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762668.874901:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ac541fa/0xdf5a7ac59f173d72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762668.874914:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ac541fa/0xdf5a7ac59f173d72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762668.888484:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762669.000134:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067217792 00000100:00100000:2.0:1789762669.000141:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067217792 00000100:00100000:2.0:1789762669.000181:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067218176 00000100:00100000:2.0:1789762669.000186:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067218176 00000100:00100000:2.0:1789762669.000222:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 100 00000100:00100000:2.0:1789762669.000232:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ebaa3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067217792:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762669.000275:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067217792:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 43us (208us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762669.000286:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 100 00000100:00100000:2.0:1789762669.000316:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 102 00000100:00100000:2.0:1789762669.000331:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ebaa2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067218176:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762669.000369:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067218176:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 41us (206us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762669.000377:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 102 00010000:00010000:2.0:1789762669.184027:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762669.686731:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967397 is committed 00000100:00100000:2.0:1789762669.689406:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067218688 00000100:00100000:2.0:1789762669.689414:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067218688 00000100:00100000:2.0:1789762669.689447:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 142922102439 time 142922192934 state 0x0 ref 1 00000100:00100000:2.0:1789762669.689456:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 143122192934 time 142922192934 state 0x1 ref 1 00000100:00100000:2.0:1789762670.075476:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 143322327677 time 143322544998 state 0x0 ref 1 00000100:00100000:2.0:1789762670.075535:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee760e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067218816:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762670.075605:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 74448896, mtime: 0x6aad9c6d, atime: 0x6aad9c5f, ctime: 0x6aad9c6d, blocks: 0x23809 00010000:00010000:2.0:1789762670.075617:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173d9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.075635:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=143s v=(0 0) 00010000:00010000:2.0:1789762670.075641:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762670.075648:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173d9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.075664:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173d9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762670.081706:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 94 00000100:00100000:2.0:1789762670.081730:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074583424:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762670.081876:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074583424:12345-0@lo:400:kworker.0 Request processed in 152us (505us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762670.081895:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 94 00000100:00100000:2.0:1789762670.082014:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074583424:0@lo:400:kworker.0 00000100:00100000:2.0:1789762670.082153:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294967322 gen 2 00000004:00080000:2.0:1789762670.082164:0:3427:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37f60a0380, transno 4294967322 00000100:00100000:2.0:1789762670.082188:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96b5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074584448:0@lo:400:kworker.0 00000100:00100000:2.0:1789762670.082475:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074583936 00000100:00100000:2.0:1789762670.082484:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074583936 00000100:00100000:2.0:1789762670.082551:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074584192 00000100:00100000:2.0:1789762670.082558:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074584192 00000100:00100000:2.0:1789762670.082657:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 103 00000100:00100000:2.0:1789762670.082687:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074583936:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762670.082802:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074583936:12345-0@lo:400:kworker.0 Request processed in 122us (1286us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762670.082869:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 103 00000100:00100000:2.0:1789762670.082899:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074583552 00000100:00100000:2.0:1789762670.082907:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074583552 00000100:00100000:2.0:1789762670.082980:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074584064 00000100:00100000:2.0:1789762670.082986:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074584064 00000100:00100000:2.0:1789762670.083045:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 105 00000100:00100000:2.0:1789762670.083068:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a37b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074583552:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762670.083217:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762670.083228:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762670.083863:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074583296 00000100:00100000:2.0:1789762670.083872:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074583296 00000100:00100000:2.0:1789762670.083982:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 107 00000100:00100000:2.0:1789762670.083998:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074583296:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762670.084069:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074583296:12345-0@lo:400:kworker.0 Request processed in 73us (565us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762670.084082:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 107 00000100:00100000:2.0:1789762670.084332:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762670.084409:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee761180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074582784:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762670.084579:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074584320 00000100:00100000:2.0:1789762670.084606:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074584320 00000100:00100000:2.0:1789762670.084671:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 44 00000100:00100000:2.0:1789762670.084723:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074584320:12345-0@lo:400:kworker.0 00002000:00010000:2.0:1789762670.108800:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:2.0:1789762670.108810:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 145417 -> 147465 00010000:00010000:2.0:1789762670.108848:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173d9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.108866:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173d9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.108878:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173d9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.108887:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80a400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762670.108893:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173d9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.108906:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173d9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.108961:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173d9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.108993:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173d9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762670.109108:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee760e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067218816:12345-192.168.203.37@tcp:4:dd.500 Request processed in 33605us (213085us total) trans 4294967399 rc 0/0 00000100:00100000:2.0:1789762670.109122:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 106 00000100:00100000:2.0:1789762670.109132:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 143322327677 time 143322544998 state 0x2 ref 0 00002000:00010000:2.0:1789762670.286844:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 75497472 -> 76546048 00002000:00010000:2.0:1789762670.286848:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 147465 -> 149513 00010000:00010000:2.0:1789762670.286858:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093f6e3bf/0xdf5a7ac59f173da3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.286866:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093f6e3bf/0xdf5a7ac59f173da3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.286872:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093f6e3bf/0xdf5a7ac59f173da3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.286877:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36cb8e2e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762670.286881:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093f6e3bf/0xdf5a7ac59f173da3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.286887:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093f6e3bf/0xdf5a7ac59f173da3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.286912:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093f6e3bf/0xdf5a7ac59f173da3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.286935:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093f6e3bf/0xdf5a7ac59f173da3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762670.286994:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067218944:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11457us (173226us total) trans 4294967400 rc 0/0 00000100:00100000:2.0:1789762670.287003:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 107 00000100:00100000:2.0:1789762670.287010:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 143522544998 time 143522592627 state 0x2 ref 0 00002000:00010000:2.0:1789762670.494303:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 76546048 -> 77594624 00002000:00010000:2.0:1789762670.494314:0:8059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789762669 -> 1789762670 00002000:00010000:2.0:1789762670.494319:0:8059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789762669 -> 1789762670 00002000:00010000:2.0:1789762670.494324:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 149513 -> 151561 00010000:00010000:2.0:1789762670.494343:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006acb03db/0xdf5a7ac59f173daa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.494365:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006acb03db/0xdf5a7ac59f173daa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.494381:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006acb03db/0xdf5a7ac59f173daa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.494395:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36cb8e3600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762670.494436:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006acb03db/0xdf5a7ac59f173daa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.494455:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006acb03db/0xdf5a7ac59f173daa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.494484:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006acb03db/0xdf5a7ac59f173daa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.494495:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006acb03db/0xdf5a7ac59f173daa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762670.494598:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067219072:12345-192.168.203.37@tcp:4:dd.500 Request processed in 18993us (204499us total) trans 4294967401 rc 0/0 00000100:00100000:2.0:1789762670.494610:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 108 00000100:00100000:2.0:1789762670.494635:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 143722592627 time 143722654900 state 0x2 ref 0 00002000:00010000:2.0:1789762670.687222:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:2.0:1789762670.687227:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 151561 -> 153609 00010000:00010000:2.0:1789762670.687237:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000125c67e3/0xdf5a7ac59f173db1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.687245:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000125c67e3/0xdf5a7ac59f173db1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.687252:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000125c67e3/0xdf5a7ac59f173db1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.687267:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36cb8e3200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762670.687270:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000125c67e3/0xdf5a7ac59f173db1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.687277:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000125c67e3/0xdf5a7ac59f173db1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.687292:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000125c67e3/0xdf5a7ac59f173db1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.687297:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000125c67e3/0xdf5a7ac59f173db1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762670.687349:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067219200:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11632us (187962us total) trans 4294967402 rc 0/0 00000100:00100000:2.0:1789762670.687357:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 109 00000100:00100000:2.0:1789762670.687363:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 143922654900 time 143922772767 state 0x2 ref 0 00000100:00100000:2.0:1789762670.689502:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067219328 00000100:00100000:2.0:1789762670.689512:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067219328 00000100:00100000:2.0:1789762670.689552:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 143922654900 time 143922772767 state 0x0 ref 1 00000100:00100000:2.0:1789762670.689561:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 144122772767 time 143922772767 state 0x1 ref 1 00002000:00010000:2.0:1789762670.887631:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:2.0:1789762670.887635:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 153609 -> 155657 00010000:00010000:2.0:1789762670.887650:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f173db8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.887660:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f173db8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.887666:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f173db8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.887671:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f1a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762670.887674:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f173db8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.887681:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f173db8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.887696:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f173db8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762670.887701:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f173db8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762670.887755:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067219328:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11965us (198301us total) trans 4294967403 rc 0/0 00000100:00100000:2.0:1789762670.887764:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 110 00000100:00100000:2.0:1789762670.887770:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 144122772767 time 144122841264 state 0x2 ref 0 00000100:00100000:2.0:1789762670.890045:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067219456 00000100:00100000:2.0:1789762670.890059:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067219456 00000100:00100000:2.0:1789762670.890091:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 144122772767 time 144122841264 state 0x0 ref 1 00000100:00100000:2.0:1789762670.890100:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 144322841264 time 144122841264 state 0x1 ref 1 00000100:00100000:2.0:1789762671.075857:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 144322841264 time 144322928967 state 0x0 ref 1 00000100:00100000:2.0:1789762671.075883:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067219456:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762671.075956:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 79691776, mtime: 0x6aad9c6e, atime: 0x6aad9c5f, ctime: 0x6aad9c6e, blocks: 0x26009 00010000:00010000:2.0:1789762671.075963:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dbf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.075977:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=144s v=(0 0) 00010000:00010000:2.0:1789762671.075981:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762671.075986:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.075997:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762671.086833:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 79691776 -> 80740352 00002000:00010000:2.0:1789762671.086838:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 155657 -> 157705 00010000:00010000:2.0:1789762671.086846:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.086855:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dbf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.086872:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.086877:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80a400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762671.086880:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.086887:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dbf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.086900:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dbf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.086906:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dbf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762671.086979:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067219456:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11107us (196967us total) trans 4294967404 rc 0/0 00000100:00100000:2.0:1789762671.086986:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 111 00000100:00100000:2.0:1789762671.086992:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 144322841264 time 144322928967 state 0x2 ref 0 00010000:00010000:2.0:1789762671.232010:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762671.275974:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 144522928967 time 144523046865 state 0x0 ref 1 00000100:00100000:2.0:1789762671.275999:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067219584:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762671.276059:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 80740352, mtime: 0x6aad9c6e, atime: 0x6aad9c5f, ctime: 0x6aad9c6e, blocks: 0x26809 00010000:00010000:2.0:1789762671.276067:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dc6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.276080:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=144s v=(0 0) 00010000:00010000:2.0:1789762671.276083:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762671.276087:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.276098:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762671.287979:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 80740352 -> 81788928 00002000:00010000:2.0:1789762671.287984:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 157705 -> 159753 00010000:00010000:2.0:1789762671.287995:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.288004:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dc6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.288010:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.288015:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80a400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762671.288019:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.288036:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dc6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.288051:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dc6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.288056:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173dc6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762671.288112:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067219584:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12124us (198757us total) trans 4294967405 rc 0/0 00000100:00100000:2.0:1789762671.288120:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 112 00000100:00100000:2.0:1789762671.288125:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 144522928967 time 144523046865 state 0x2 ref 0 00000001:00080000:2.0:1789762671.488613:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967406 is committed 00000100:00100000:2.0:1789762671.491014:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067219840 00000100:00100000:2.0:1789762671.491023:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067219840 00000100:00100000:2.0:1789762671.491057:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 144723046865 time 144723108709 state 0x0 ref 1 00000100:00100000:2.0:1789762671.491065:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 144923108709 time 144723108709 state 0x1 ref 1 00000001:00080000:2.0:1789762671.689091:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967407 is committed 00000100:00100000:2.0:1789762671.691817:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067219968 00000100:00100000:2.0:1789762671.691825:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067219968 00000100:00100000:2.0:1789762671.691861:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 144923108709 time 144923208644 state 0x0 ref 1 00000100:00100000:2.0:1789762671.691870:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 145123208644 time 144923208644 state 0x1 ref 1 00000100:00100000:2.0:1789762671.876248:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 145123208644 time 145123316498 state 0x0 ref 1 00000100:00100000:2.0:1789762671.876304:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067219968:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762671.876364:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 83886080, mtime: 0x6aad9c6f, atime: 0x6aad9c5f, ctime: 0x6aad9c6f, blocks: 0x28009 00010000:00010000:2.0:1789762671.876371:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173ddb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.876392:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=145s v=(0 0) 00010000:00010000:2.0:1789762671.876396:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762671.876400:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173ddb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.876413:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173ddb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762671.892909:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 83886080 -> 84934656 00002000:00010000:2.0:1789762671.892943:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 163849 -> 165897 00010000:00010000:2.0:1789762671.892959:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173ddb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.892971:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173ddb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.892980:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173ddb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.893001:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80a400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762671.893006:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173ddb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.893014:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173ddb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.893033:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173ddb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762671.893039:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173ddb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762671.893122:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067219968:12345-192.168.203.37@tcp:4:dd.500 Request processed in 16849us (201353us total) trans 4294967408 rc 0/0 00000100:00100000:2.0:1789762671.893134:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 115 00000100:00100000:2.0:1789762671.893142:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 145123208644 time 145123316498 state 0x2 ref 0 00000100:00100000:2.0:1789762672.076373:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 145323316498 time 145323444892 state 0x0 ref 1 00000100:00100000:2.0:1789762672.076420:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067220096:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762672.076482:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 84934656, mtime: 0x6aad9c6f, atime: 0x6aad9c5f, ctime: 0x6aad9c6f, blocks: 0x28809 00010000:00010000:2.0:1789762672.076490:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173de2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.076521:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=145s v=(0 0) 00010000:00010000:2.0:1789762672.076525:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762672.076533:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173de2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.076547:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173de2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762672.088262:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 84934656 -> 85983232 00002000:00010000:2.0:1789762672.088268:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 165897 -> 167945 00010000:00010000:2.0:1789762672.088278:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173de2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.088297:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173de2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.088304:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173de2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.088309:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80a400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762672.088312:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173de2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.088319:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173de2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.088346:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173de2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.088351:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173de2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762672.088433:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067220096:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12038us (191685us total) trans 4294967409 rc 0/0 00000100:00100000:2.0:1789762672.088443:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 116 00000100:00100000:2.0:1789762672.088449:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 145323316498 time 145323444892 state 0x2 ref 0 00000100:00100000:2.0:1789762672.290889:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067220352 00000100:00100000:2.0:1789762672.290898:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067220352 00000100:00100000:2.0:1789762672.290958:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 145523444892 time 145523510349 state 0x0 ref 1 00000100:00100000:2.0:1789762672.290970:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 145723510349 time 145523510349 state 0x1 ref 1 00000100:00100000:2.0:1789762672.491028:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067220480 00000100:00100000:2.0:1789762672.491036:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067220480 00000100:00100000:2.0:1789762672.491069:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 145723510349 time 145723593634 state 0x0 ref 1 00000100:00100000:2.0:1789762672.491078:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 145923593634 time 145723593634 state 0x1 ref 1 00000100:00100000:2.0:1789762672.676640:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 145923593634 time 145923692572 state 0x0 ref 1 00000100:00100000:2.0:1789762672.676676:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ebaa3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067220480:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762672.676737:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 88080384, mtime: 0x6aad9c70, atime: 0x6aad9c5f, ctime: 0x6aad9c70, blocks: 0x2a009 00010000:00010000:2.0:1789762672.676762:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173df7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.676780:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=145s v=(0 0) 00010000:00010000:2.0:1789762672.676785:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762672.676791:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173df7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.676830:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173df7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762672.876734:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 146123692572 time 146123805820 state 0x0 ref 1 00000100:00100000:2.0:1789762672.876764:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067220608:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762672.876833:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 89128960, mtime: 0x6aad9c70, atime: 0x6aad9c5f, ctime: 0x6aad9c70, blocks: 0x2a809 00010000:00010000:2.0:1789762672.876843:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8551867/0xdf5a7ac59f173dfe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.876861:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=146s v=(0 0) 00010000:00010000:2.0:1789762672.876866:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762672.876872:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8551867/0xdf5a7ac59f173dfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.876886:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8551867/0xdf5a7ac59f173dfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762672.891902:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762673.152219:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074584704 00000100:00100000:2.0:1789762673.152226:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074584704 00000100:00100000:2.0:1789762673.152274:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 96 00000100:00100000:2.0:1789762673.152284:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6096300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074584704:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762673.152322:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762673.152370:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6096300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074584704:12345-0@lo:41:osp-pre-0-1.0 Request processed in 88us (193us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762673.152376:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 96 00010000:00010000:2.0:1789762673.280016:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762673.293702:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067220992 00000100:00100000:2.0:1789762673.293714:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067220992 00000100:00100000:2.0:1789762673.293759:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 146523897192 time 146523995674 state 0x0 ref 1 00000100:00100000:2.0:1789762673.293774:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 146723995674 time 146523995674 state 0x1 ref 1 00000100:00100000:2.0:1789762673.498516:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762673.677232:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 146924178314 time 146924303375 state 0x0 ref 1 00000100:00100000:2.0:1789762673.677264:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067221120:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762673.677317:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 93323264, mtime: 0x6aad9c71, atime: 0x6aad9c5f, ctime: 0x6aad9c71, blocks: 0x2c809 00010000:00010000:2.0:1789762673.677325:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ce625400/0xdf5a7ac59f173e1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762673.677339:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=146s v=(0 0) 00010000:00010000:2.0:1789762673.677350:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762673.677356:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce625400/0xdf5a7ac59f173e1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762673.677386:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce625400/0xdf5a7ac59f173e1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762673.694169:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 93323264 -> 94371840 00002000:00010000:2.0:1789762673.694180:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 182281 -> 184331 00010000:00010000:2.0:1789762673.694198:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ce625400/0xdf5a7ac59f173e1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762673.694217:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce625400/0xdf5a7ac59f173e1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762673.694230:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce625400/0xdf5a7ac59f173e1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762673.694240:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff80b600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762673.694246:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce625400/0xdf5a7ac59f173e1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762673.694257:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce625400/0xdf5a7ac59f173e1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762673.694279:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce625400/0xdf5a7ac59f173e1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762673.694289:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce625400/0xdf5a7ac59f173e1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762673.694408:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067221120:12345-192.168.203.37@tcp:4:dd.500 Request processed in 17156us (195891us total) trans 4294967417 rc 0/0 00000100:00100000:2.0:1789762673.694418:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 124 00000100:00100000:2.0:1789762673.694426:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 146924178314 time 146924303375 state 0x2 ref 0 00000100:00100000:2.0:1789762673.905450:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067221376 00000100:00100000:2.0:1789762673.905465:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067221376 00000100:00100000:2.0:1789762673.905523:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 147124303375 time 147124410269 state 0x0 ref 1 00000100:00100000:2.0:1789762673.905539:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 147324410269 time 147124410269 state 0x1 ref 1 00000100:00100000:2.0:1789762674.096592:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762674.119935:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067221632 00000100:00100000:2.0:1789762674.119942:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067221632 00000100:00100000:2.0:1789762674.119997:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 283 00000100:00100000:2.0:1789762674.120011:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1578c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067221632:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762674.120068:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1578c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067221632:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 60us (194us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762674.120076:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 283 00000100:00100000:2.0:1789762674.120141:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 111 00000100:00100000:2.0:1789762674.120150:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067221760:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762674.120184:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067221760:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 31us (160us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762674.120209:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 111 00000100:00100000:2.0:1789762674.120231:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067222272 00000100:00100000:2.0:1789762674.120243:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067222272 00000100:00100000:2.0:1789762674.277606:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 147524561223 time 147524677590 state 0x0 ref 1 00000100:00100000:2.0:1789762674.277631:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067221504:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762674.277681:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 96468992, mtime: 0x6aad9c71, atime: 0x6aad9c5f, ctime: 0x6aad9c71, blocks: 0x2e00b 00010000:00010000:2.0:1789762674.277688:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1326af5/0xdf5a7ac59f173e2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762674.277701:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=147s v=(0 0) 00010000:00010000:2.0:1789762674.277705:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762674.277709:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1326af5/0xdf5a7ac59f173e2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762674.277720:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1326af5/0xdf5a7ac59f173e2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762674.291544:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762674.492184:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067222528 00000100:00100000:2.0:1789762674.492192:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067222528 00000100:00100000:2.0:1789762674.492227:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 147724677590 time 147724744292 state 0x0 ref 1 00000100:00100000:2.0:1789762674.492236:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 147924744292 time 147724744292 state 0x1 ref 1 00000100:00100000:2.0:1789762674.691413:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067222656 00000100:00100000:2.0:1789762674.691422:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067222656 00000100:00100000:2.0:1789762674.691453:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 147924744292 time 147924876709 state 0x0 ref 1 00000100:00100000:2.0:1789762674.691462:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 148124876709 time 147924876709 state 0x1 ref 1 00000100:00100000:2.0:1789762674.877892:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 148124876709 time 148124964102 state 0x0 ref 1 00000100:00100000:2.0:1789762674.877949:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067222656:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762674.878001:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 99614720, mtime: 0x6aad9c72, atime: 0x6aad9c5f, ctime: 0x6aad9c72, blocks: 0x2f80b 00010000:00010000:2.0:1789762674.878009:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8551867/0xdf5a7ac59f173e44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762674.878024:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=148s v=(0 0) 00010000:00010000:2.0:1789762674.878027:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762674.878032:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8551867/0xdf5a7ac59f173e44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762674.878045:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8551867/0xdf5a7ac59f173e44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762675.077986:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 148324964102 time 148325058137 state 0x0 ref 1 00000100:00100000:2.0:1789762675.078014:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067222784:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762675.078063:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 100663296, mtime: 0x6aad9c72, atime: 0x6aad9c5f, ctime: 0x6aad9c72, blocks: 0x3000b 00010000:00010000:2.0:1789762675.078070:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173e4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762675.078083:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=148s v=(0 0) 00010000:00010000:2.0:1789762675.078087:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762675.078091:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173e4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762675.078116:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017fc84e3/0xdf5a7ac59f173e4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762675.200598:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074585728 00000100:00100000:2.0:1789762675.200605:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074585728 00000100:00100000:2.0:1789762675.200668:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074585472 00000100:00100000:2.0:1789762675.200673:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074585472 00000100:00100000:2.0:1789762675.200720:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 97 00000100:00100000:2.0:1789762675.200734:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074585728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762675.200814:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074585728:12345-0@lo:400:kworker.0 Request processed in 83us (424us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762675.200832:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 97 00000100:00100000:2.0:1789762675.200883:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074586112 00000100:00100000:2.0:1789762675.200898:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074586112 00000100:00100000:2.0:1789762675.200956:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 113 00000100:00100000:2.0:1789762675.200967:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6097b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074585600:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762675.201006:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6097b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074585600:12345-0@lo:400:kworker.0 Request processed in 40us (676us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762675.201023:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 113 00000100:00100000:2.0:1789762675.201047:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 115 00000100:00100000:2.0:1789762675.201061:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074586240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762675.201103:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074586240:12345-0@lo:400:kworker.0 Request processed in 49us (612us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762675.201113:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 115 00000100:00100000:2.0:1789762675.201141:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074584832 00000100:00100000:2.0:1789762675.201153:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074584832 00000100:00100000:2.0:1789762675.201210:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:2.0:1789762675.201223:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074584832:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789762675.201236:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 236650496 pending 0 free 3765436416 avail 3755999232 00000100:00100000:2.0:1789762675.201281:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074584832:12345-0@lo:13:osp-pre-0-1.0 Request processed in 58us (158us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762675.201294:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00000100:00100000:2.0:1789762675.201332:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074586624 00000100:00100000:2.0:1789762675.201337:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074586624 00000100:00100000:2.0:1789762675.201371:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074586368 00000100:00100000:2.0:1789762675.201376:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074586368 00000100:00100000:2.0:1789762675.201416:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 48 00000100:00100000:2.0:1789762675.201431:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074586496:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762675.201501:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074586496:12345-0@lo:400:kworker.0 Request processed in 72us (242us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762675.201517:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 48 00000100:00100000:2.0:1789762675.201561:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074585984 00000100:00100000:2.0:1789762675.201565:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074585984 00000100:00100000:2.0:1789762675.201600:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074585344 00000100:00100000:2.0:1789762675.201603:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074585344 00000100:00100000:2.0:1789762675.201633:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 116 00000100:00100000:2.0:1789762675.201643:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074585984:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762675.201688:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074585984:12345-0@lo:400:kworker.0 Request processed in 46us (541us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762675.201694:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 116 00000100:00100000:2.0:1789762675.201715:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 118 00000100:00100000:2.0:1789762675.201729:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6097100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074585856:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762675.201791:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6097100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074585856:12345-0@lo:400:kworker.0 Request processed in 56us (485us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762675.201806:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 118 00000100:00100000:2.0:1789762675.292398:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067223040 00000100:00100000:2.0:1789762675.292409:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067223040 00000100:00100000:2.0:1789762675.292447:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 148525058137 time 148525113729 state 0x0 ref 1 00000100:00100000:2.0:1789762675.292456:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 148725113729 time 148525113729 state 0x1 ref 1 00010000:00010000:2.0:1789762675.328018:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762675.493474:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067223168 00000100:00100000:2.0:1789762675.493482:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067223168 00000100:00100000:2.0:1789762675.493518:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 148725113729 time 148725227635 state 0x0 ref 1 00000100:00100000:2.0:1789762675.493528:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 148925227635 time 148725227635 state 0x1 ref 1 00000100:00100000:2.0:1789762675.678208:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 148925227635 time 148925280057 state 0x0 ref 1 00000100:00100000:2.0:1789762675.678238:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067223168:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762675.678290:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 103809024, mtime: 0x6aad9c73, atime: 0x6aad9c5f, ctime: 0x6aad9c73, blocks: 0x3180b 00010000:00010000:2.0:1789762675.678298:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000862250ac/0xdf5a7ac59f173e60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762675.678311:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=148s v=(0 0) 00010000:00010000:2.0:1789762675.678315:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762675.678320:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000862250ac/0xdf5a7ac59f173e60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762675.678331:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000862250ac/0xdf5a7ac59f173e60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762675.691251:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067223424 00000100:00100000:2.0:1789762675.691260:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067223424 00000100:00100000:2.0:1789762675.691298:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 18 00000100:00100000:2.0:1789762675.691305:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfce300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:11414:x1876702067223424:12345-192.168.203.37@tcp:35:dd.500 00000001:00080000:2.0:1789762675.691529:0:13186:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967329 00000100:00100000:2.0:1789762675.691595:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfce300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:11414:x1876702067223424:12345-192.168.203.37@tcp:35:dd.500 Request processed in 290us (393us total) trans 4294967330 rc 0/0 00000100:00100000:2.0:1789762675.691601:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 18 00000020:00080000:2.0:1789762675.691604:0:13186:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789762675.878309:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 149125280057 time 149125380655 state 0x0 ref 1 00000100:00100000:2.0:1789762675.878339:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067223936:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762675.878380:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173e91 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762675.878394:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762675.878398:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762675.878403:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173e91 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762675.878415:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067f495cc/0xdf5a7ac59f173e91 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762676.084407:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067224192 00000100:00100000:2.0:1789762676.084416:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067224192 00000100:00100000:2.0:1789762676.084454:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 149325380655 time 149325455324 state 0x0 ref 1 00000100:00100000:2.0:1789762676.084463:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 149525455324 time 149325455324 state 0x1 ref 1 00000001:00080000:2.0:1789762676.230574:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967331 is committed 00000100:00100000:2.0:1789762676.283651:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067224320 00000100:00100000:2.0:1789762676.283661:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067224320 00000100:00100000:2.0:1789762676.283701:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 149525455324 time 149525529075 state 0x0 ref 1 00000100:00100000:2.0:1789762676.283714:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 149725529075 time 149525529075 state 0x1 ref 1 00000100:00100000:2.0:1789762676.478512:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 149725529075 time 149725584240 state 0x0 ref 1 00000100:00100000:2.0:1789762676.478540:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067224320:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762676.478577:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ea6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.478590:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762676.478593:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762676.478598:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ea6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.478609:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ea6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.481665:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ea6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.481679:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ea6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.481701:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ea6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.481710:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762676.481716:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ea6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.481727:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ea6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.481745:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ea6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.481754:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ea6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762676.481820:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067224320:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3292us (198224us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762676.481831:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 138 00000100:00100000:2.0:1789762676.481837:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 149725529075 time 149725584240 state 0x2 ref 0 00000100:00100000:2.0:1789762676.678613:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 149925584240 time 149925685495 state 0x0 ref 1 00000100:00100000:2.0:1789762676.678643:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067224448:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762676.678680:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ead lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.678710:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762676.678715:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762676.678722:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ead lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.678737:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ead lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.681699:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ead lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.681709:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ead lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.681715:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ead lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.681721:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762676.681725:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ead lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.681731:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ead lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.681744:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ead lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762676.681760:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173ead lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762676.681817:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067224448:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3188us (197924us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762676.681825:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 139 00000100:00100000:2.0:1789762676.681830:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 149925584240 time 149925685495 state 0x2 ref 0 00000100:00100000:2.0:1789762677.084522:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067224832 00000100:00100000:2.0:1789762677.084530:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067224832 00000100:00100000:2.0:1789762677.084575:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 150325742450 time 150325804047 state 0x0 ref 1 00000100:00100000:2.0:1789762677.084583:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 150525804047 time 150325804047 state 0x1 ref 1 00000100:00100000:2.0:1789762677.248229:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074587008 00000100:00100000:2.0:1789762677.248239:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074587008 00000100:00100000:2.0:1789762677.248291:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 99 00000100:00100000:2.0:1789762677.248301:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfcea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074587008:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762677.248336:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762677.248380:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfcea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074587008:12345-0@lo:41:osp-pre-1-0.0 Request processed in 81us (194us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762677.248388:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 99 00000100:00100000:2.0:1789762677.284264:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067224960 00000100:00100000:2.0:1789762677.284272:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067224960 00000100:00100000:2.0:1789762677.284310:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 150525804047 time 150525891960 state 0x0 ref 1 00000100:00100000:2.0:1789762677.284321:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 150725891960 time 150525891960 state 0x1 ref 1 00010000:00010000:2.0:1789762677.376034:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762677.478907:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 150725891960 time 150725976580 state 0x0 ref 1 00000100:00100000:2.0:1789762677.478963:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067224960:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762677.479008:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ec9 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.479028:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762677.479033:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762677.479040:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ec9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.479055:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ec9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.481889:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ec9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.481900:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ec9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.481907:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ec9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.481913:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762677.481937:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ec9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.481955:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ec9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.481971:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ec9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.481977:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ec9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762677.482026:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067224960:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3080us (197814us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762677.482033:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 143 00000100:00100000:2.0:1789762677.482038:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 150725891960 time 150725976580 state 0x2 ref 0 00000100:00100000:2.0:1789762677.483875:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762677.678996:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 150925976580 time 150926065140 state 0x0 ref 1 00000100:00100000:2.0:1789762677.679032:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067225088:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762677.679070:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ed0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.679084:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762677.679088:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762677.679092:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ed0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.679120:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f173ed0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762677.684061:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067225216 00000100:00100000:2.0:1789762677.684069:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067225216 00000100:00100000:2.0:1789762677.684099:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 150925976580 time 150926065140 state 0x0 ref 1 00000100:00100000:2.0:1789762677.684108:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 151126065140 time 150926065140 state 0x1 ref 1 00010000:00010000:2.0:1789762677.882225:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f98264d4/0xdf5a7ac59f173ed7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.882242:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f98264d4/0xdf5a7ac59f173ed7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.882249:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f98264d4/0xdf5a7ac59f173ed7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.882254:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37fdfcb600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762677.882258:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f98264d4/0xdf5a7ac59f173ed7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.882264:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f98264d4/0xdf5a7ac59f173ed7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.882276:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f98264d4/0xdf5a7ac59f173ed7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762677.882289:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f98264d4/0xdf5a7ac59f173ed7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762677.882340:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6094e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067225216:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3223us (198355us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762677.882357:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 145 00000100:00100000:2.0:1789762677.882363:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 151126065140 time 151126168435 state 0x2 ref 0 00000100:00100000:2.0:1789762678.079237:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 151326168435 time 151326303227 state 0x0 ref 1 00000100:00100000:2.0:1789762678.079311:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067225344:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762678.079375:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ede lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.079403:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762678.079410:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762678.079419:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ede lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.079438:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ede lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.084876:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ede lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.084970:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ede lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.084985:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ede lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.084996:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762678.085005:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ede lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.085019:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ede lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.085045:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ede lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.085056:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ede lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762678.085141:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067225344:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5873us (200943us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762678.085157:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 146 00000100:00100000:2.0:1789762678.085167:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 151326168435 time 151326303227 state 0x2 ref 0 00000100:00100000:2.0:1789762678.287048:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067225600 00000100:00100000:2.0:1789762678.287070:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067225600 00000100:00100000:2.0:1789762678.287120:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 151526303227 time 151526388142 state 0x0 ref 1 00000100:00100000:2.0:1789762678.287132:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 151726388142 time 151526388142 state 0x1 ref 1 00000100:00100000:2.0:1789762678.486385:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067225728 00000100:00100000:2.0:1789762678.486399:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067225728 00000100:00100000:2.0:1789762678.486456:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 151726388142 time 151726577200 state 0x0 ref 1 00000100:00100000:2.0:1789762678.486477:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 151926577200 time 151726577200 state 0x1 ref 1 00000100:00100000:2.0:1789762678.679596:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 151926577200 time 151926668504 state 0x0 ref 1 00000100:00100000:2.0:1789762678.679626:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067225728:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762678.679662:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ef3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.679676:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762678.679679:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762678.679684:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ef3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.679695:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ef3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.682390:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ef3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.682401:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ef3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.682418:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ef3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.682424:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762678.682427:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ef3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.682433:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ef3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.682446:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ef3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.682451:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173ef3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762678.682497:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067225728:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2886us (196196us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762678.682504:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 149 00000100:00100000:2.0:1789762678.682510:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 151926577200 time 151926668504 state 0x2 ref 0 00000100:00100000:2.0:1789762678.879717:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 152126668504 time 152126789372 state 0x0 ref 1 00000100:00100000:2.0:1789762678.879747:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067225856:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762678.879797:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173efa lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.879824:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762678.879828:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762678.879833:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173efa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.879846:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173efa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.882670:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173efa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.882680:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173efa lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.882687:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173efa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.882693:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762678.882697:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173efa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.882703:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173efa lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.882716:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173efa lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762678.882732:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173efa lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762678.882796:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067225856:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3064us (198217us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762678.882802:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 150 00000100:00100000:2.0:1789762678.882806:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 152126668504 time 152126789372 state 0x2 ref 0 00000100:00100000:2.0:1789762679.085060:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067226112 00000100:00100000:2.0:1789762679.085068:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067226112 00000100:00100000:2.0:1789762679.085101:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 152326789372 time 152326861999 state 0x0 ref 1 00000100:00100000:2.0:1789762679.085110:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 152526861999 time 152326861999 state 0x1 ref 1 00000100:00100000:2.0:1789762679.240106:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067226368 00000100:00100000:2.0:1789762679.240113:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067226368 00000100:00100000:2.0:1789762679.240175:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 119 00000100:00100000:2.0:1789762679.240192:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2397:x1876702067226368:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762679.240279:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2397:x1876702067226368:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 92us (321us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762679.240287:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 119 00000100:00100000:2.0:1789762679.242132:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067226240 00000100:00100000:2.0:1789762679.242138:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067226240 00000100:00100000:2.0:1789762679.242179:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 285 00000100:00100000:2.0:1789762679.242191:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d157bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067226240:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762679.242231:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d157bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067226240:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 44us (178us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762679.242258:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 285 00000100:00100000:2.0:1789762679.244025:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762679.244064:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00080000:2.0:1789762679.244135:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762679.244142:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762679.284659:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067227136 00000100:00100000:2.0:1789762679.284668:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067227136 00000100:00100000:2.0:1789762679.284703:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 152526861999 time 152526937212 state 0x0 ref 1 00000100:00100000:2.0:1789762679.284715:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 152726937212 time 152526937212 state 0x1 ref 1 00000100:00100000:2.0:1789762679.479968:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 152726937212 time 152727038417 state 0x0 ref 1 00000100:00100000:2.0:1789762679.480002:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067227136:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762679.480041:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f0f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.480058:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762679.480063:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762679.480071:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f0f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.480086:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f0f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.483013:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f0f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.483041:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f0f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.483048:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f0f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.483053:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762679.483057:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f0f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.483063:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f0f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.483077:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f0f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.483082:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f0f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762679.483138:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067227136:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3151us (198526us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762679.483145:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 153 00000100:00100000:2.0:1789762679.483151:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 152726937212 time 152727038417 state 0x2 ref 0 00000100:00100000:2.0:1789762679.680060:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 152927038417 time 152927132037 state 0x0 ref 1 00000100:00100000:2.0:1789762679.680103:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfc7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067227264:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762679.680138:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f16 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.680152:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762679.680155:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762679.680160:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f16 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.680171:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f16 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.682973:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f16 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.682984:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f16 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.682990:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f16 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.682996:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762679.683000:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f16 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.683006:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f16 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.683030:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f16 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762679.683035:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f16 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762679.683081:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfc7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067227264:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2990us (197455us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762679.683088:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 154 00000100:00100000:2.0:1789762679.683093:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 152927038417 time 152927132037 state 0x2 ref 0 00000100:00080000:2.0:1789762680.080320:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762680.085514:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067227648 00000100:00100000:2.0:1789762680.085522:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067227648 00000100:00100000:2.0:1789762680.085552:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 153327187905 time 153327303487 state 0x0 ref 1 00000100:00100000:2.0:1789762680.085561:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 153527303487 time 153327303487 state 0x1 ref 1 00000100:00100000:2.0:1789762680.285504:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067227776 00000100:00100000:2.0:1789762680.285513:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067227776 00000100:00100000:2.0:1789762680.285547:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 153527303487 time 153527364305 state 0x0 ref 1 00000100:00100000:2.0:1789762680.285556:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 153727364305 time 153527364305 state 0x1 ref 1 00000100:00100000:2.0:1789762680.320277:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074587648 00000100:00100000:2.0:1789762680.320286:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074587648 00000100:00100000:2.0:1789762680.320340:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074587520 00000100:00100000:2.0:1789762680.320346:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074587520 00000100:00100000:2.0:1789762680.320389:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 54 00000100:00100000:2.0:1789762680.320403:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074587648:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789762680.320441:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762680.320498:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074587648:12345-0@lo:13:osp-pre-1-1.0 Request processed in 99us (298us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762680.320515:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 54 00000100:00100000:2.0:1789762680.320542:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 55 00000100:00100000:2.0:1789762680.320552:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074587520:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762680.320563:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762680.320597:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074587520:12345-0@lo:13:osp-pre-1-0.0 Request processed in 47us (326us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762680.320604:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 55 00000100:00100000:2.0:1789762680.320653:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074589184 00000100:00100000:2.0:1789762680.320659:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074589184 00000100:00100000:2.0:1789762680.320708:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 51 00000100:00100000:2.0:1789762680.320724:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee70f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074589184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762680.320788:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee70f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074589184:12345-0@lo:400:kworker.0 Request processed in 67us (286us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762680.320801:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 51 00000100:00100000:2.0:1789762680.320828:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074589056 00000100:00100000:2.0:1789762680.320833:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074589056 00000100:00100000:2.0:1789762680.320868:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:2.0:1789762680.320880:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074589056:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762680.320958:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074589056:12345-0@lo:400:kworker.0 Request processed in 79us (157us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762680.320979:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00000100:00100000:2.0:1789762680.321033:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762680.321062:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff615500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074588416:0@lo:400:kworker.0 00000100:00100000:2.0:1789762680.321108:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762680.321135:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff617480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074588928:0@lo:400:kworker.0 00000100:00100000:2.0:1789762680.321182:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762680.321240:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074587520:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762680.321262:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074587904:0@lo:400:kworker.0 00000100:00100000:2.0:1789762680.321282:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff615500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074588416:0@lo:400:kworker.0 00000100:00100000:2.0:1789762680.321323:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff617480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074588928:0@lo:400:kworker.0 00000100:00100000:2.0:1789762680.321352:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074589312 00000100:00100000:2.0:1789762680.321359:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074589312 00000100:00100000:2.0:1789762680.321395:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 54 00000100:00100000:2.0:1789762680.321406:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074589312:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762680.321459:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074589312:12345-0@lo:400:kworker.0 Request processed in 56us (182us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762680.321467:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 54 00000100:00100000:2.0:1789762680.321553:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074588032:0@lo:400:kworker.0 00010000:00010000:2.0:1789762680.448031:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762680.480375:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 153727364305 time 153727447702 state 0x0 ref 1 00000100:00100000:2.0:1789762680.480406:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067227776:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762680.480444:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f32 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.480474:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762680.480478:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762680.480483:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f32 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.480493:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f32 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.483377:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f32 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.483386:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f32 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.483392:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f32 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.483398:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762680.483401:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f32 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.483407:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f32 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.483421:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f32 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.483436:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f32 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762680.483482:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067227776:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3092us (198062us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762680.483489:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 158 00000100:00100000:2.0:1789762680.483494:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 153727364305 time 153727447702 state 0x2 ref 0 00000100:00100000:2.0:1789762680.680461:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 153927447702 time 153927533037 state 0x0 ref 1 00000100:00100000:2.0:1789762680.680490:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067227904:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762680.680526:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f39 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.680540:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762680.680543:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762680.680548:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f39 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.680559:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f39 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.683501:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f39 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.683523:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f39 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.683529:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f39 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.683534:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762680.683538:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f39 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.683544:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f39 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.683557:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f39 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762680.683562:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f39 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762680.683609:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067227904:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3134us (198048us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762680.683616:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 159 00000100:00100000:2.0:1789762680.683622:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 153927447702 time 153927533037 state 0x2 ref 0 00000100:00100000:2.0:1789762681.085670:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067228288 00000100:00100000:2.0:1789762681.085684:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067228288 00000100:00100000:2.0:1789762681.085715:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 154327587955 time 154327654029 state 0x0 ref 1 00000100:00100000:2.0:1789762681.085723:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 154527654029 time 154327654029 state 0x1 ref 1 00000100:00100000:2.0:1789762681.286896:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067228416 00000100:00100000:2.0:1789762681.286905:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067228416 00000100:00100000:2.0:1789762681.286977:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 154527654029 time 154527746772 state 0x0 ref 1 00000100:00100000:2.0:1789762681.286987:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 154727746772 time 154527746772 state 0x1 ref 1 00000100:00100000:2.0:1789762681.480795:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 154727746772 time 154727867847 state 0x0 ref 1 00000100:00100000:2.0:1789762681.480823:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff862300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067228416:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762681.480861:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f55 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.480875:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762681.480878:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762681.480883:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f55 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.480893:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f55 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.483689:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f55 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.483699:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f55 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.483716:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f55 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.483722:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762681.483725:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f55 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.483745:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f55 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.483759:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f55 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.483765:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f55 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762681.483816:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff862300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067228416:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3007us (196980us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762681.483822:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 163 00000100:00100000:2.0:1789762681.483828:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 154727746772 time 154727867847 state 0x2 ref 0 00000100:00100000:2.0:1789762681.680888:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 154927867847 time 154927959072 state 0x0 ref 1 00000100:00100000:2.0:1789762681.680935:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067228544:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762681.680977:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f5c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.681008:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762681.681013:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762681.681020:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f5c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.681035:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f5c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.683902:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f5c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.683911:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f5c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.683942:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f5c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.683948:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762681.683953:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f5c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.683959:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f5c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.683976:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f5c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762681.683999:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f173f5c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762681.684057:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067228544:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3153us (198414us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762681.684065:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 164 00000100:00100000:2.0:1789762681.684071:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 154927867847 time 154927959072 state 0x2 ref 0 00000100:00100000:2.0:1789762682.086251:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762682.286977:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762682.368091:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff861c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074589440:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762682.368151:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762682.368394:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff861c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074589440:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762682.486399:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067229184 00000100:00100000:2.0:1789762682.486408:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067229184 00000100:00100000:2.0:1789762682.486442:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 155728216177 time 155728334139 state 0x0 ref 1 00000100:00100000:2.0:1789762682.486451:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 155928334139 time 155728334139 state 0x1 ref 1 00000100:00100000:2.0:1789762682.687247:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067229312 00000100:00100000:2.0:1789762682.687255:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067229312 00000100:00100000:2.0:1789762682.687290:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 155928334139 time 155928459687 state 0x0 ref 1 00000100:00100000:2.0:1789762682.687302:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 156128459687 time 155928459687 state 0x1 ref 1 00000020:00000080:2.0:1789762682.889036:0:36:0:(genops.c:844:class_export_destroy()) destroying export ffff9c37ee89e000/fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 for lustre-OST0001 00000020:00000080:2.0:1789762682.889078:0:36:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9c37ee89f000 for lustre-OST0001 00000100:00100000:2.0:1789762683.081566:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 156328540184 time 156328637955 state 0x0 ref 1 00000100:00100000:2.0:1789762683.081595:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6095880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067229440:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762683.081632:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f173f8d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762683.081646:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762683.081649:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762683.081653:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f173f8d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762683.081665:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f173f8d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762683.084623:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f173f8d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762683.084632:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f173f8d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762683.084637:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f173f8d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762683.084642:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762683.084646:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f173f8d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762683.084667:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f173f8d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762683.084680:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f173f8d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762683.084685:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f173f8d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762683.084751:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6095880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067229440:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3171us (197644us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762683.084759:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 171 00000100:00100000:2.0:1789762683.084764:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 156328540184 time 156328637955 state 0x2 ref 0 00000100:00100000:2.0:1789762683.687579:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067229952 00000100:00100000:2.0:1789762683.687589:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067229952 00000100:00100000:2.0:1789762683.687633:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 156928783569 time 156928940744 state 0x0 ref 1 00000100:00100000:2.0:1789762683.687644:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 157128940744 time 156928940744 state 0x1 ref 1 00000100:00100000:2.0:1789762683.887181:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762684.082117:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 157329068572 time 157329182731 state 0x0 ref 1 00000100:00100000:2.0:1789762684.082175:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067230080:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762684.082234:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c10cae0e/0xdf5a7ac59f173fb0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762684.082280:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762684.082285:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762684.082292:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c10cae0e/0xdf5a7ac59f173fb0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762684.082306:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c10cae0e/0xdf5a7ac59f173fb0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762684.359993:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067230464 00000100:00100000:2.0:1789762684.360007:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067230464 00000100:00100000:2.0:1789762684.360080:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 287 00000100:00100000:2.0:1789762684.360093:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d157b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067230464:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762684.360177:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d157b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067230464:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 86us (276us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762684.360188:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 287 00000100:00100000:2.0:1789762684.360233:0:6419:0:(service.c:3462:ptlrpc_start_thread()) mdt[0] started 5 min 3 max 116 00000100:00100000:2.0:1789762684.360247:0:6419:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt00_005' 00010000:00010000:2.0:1789762684.544019:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762684.687516:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067231232 00000100:00100000:2.0:1789762684.687527:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067231232 00000100:00100000:2.0:1789762684.687580:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 157929307185 time 157929412674 state 0x0 ref 1 00000100:00100000:2.0:1789762684.687592:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 158129412674 time 157929412674 state 0x1 ref 1 00000100:00100000:2.0:1789762685.082640:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 158329544582 time 158329708552 state 0x0 ref 1 00000100:00100000:2.0:1789762685.082677:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1772d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067231360:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762685.082859:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173fd3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762685.082877:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762685.082881:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762685.082888:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173fd3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762685.082902:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173fd3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762685.087036:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173fd3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762685.087052:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173fd3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762685.087059:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173fd3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762685.087065:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762685.087068:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173fd3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762685.087075:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173fd3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762685.087114:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173fd3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762685.087124:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f173fd3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762685.087186:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1772d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067231360:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4523us (198937us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762685.087196:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 181 00000100:00100000:2.0:1789762685.087203:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 158329544582 time 158329708552 state 0x2 ref 0 00000100:00100000:2.0:1789762685.089549:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762685.440563:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1770700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074589952:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762685.440633:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762685.440662:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f60a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074590464:0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.440698:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762685.440755:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c8171500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074590976:0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.440804:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762685.441128:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074590848 00000100:00100000:2.0:1789762685.441138:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074590848 00000100:00100000:2.0:1789762685.441181:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074591232 00000100:00100000:2.0:1789762685.441186:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074591232 00000100:00100000:2.0:1789762685.441233:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 133 00000100:00100000:2.0:1789762685.441252:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ca317b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074590336:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.441346:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ca317b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074590336:12345-0@lo:400:kworker.0 Request processed in 97us (878us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762685.441371:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 133 00000100:00100000:2.0:1789762685.441405:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1789762685.441428:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ca317480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074590848:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.441513:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ca317480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074590848:12345-0@lo:400:kworker.0 Request processed in 96us (963us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762685.441520:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1789762685.441558:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:2.0:1789762685.441583:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c8171f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074590976:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.441650:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c8171f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074590976:12345-0@lo:400:kworker.0 Request processed in 64us (838us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762685.441679:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00000100:00100000:2.0:1789762685.441740:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 106 00000100:00100000:2.0:1789762685.441753:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c8173800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074590464:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.441810:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c8173800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074590464:12345-0@lo:400:kworker.0 Request processed in 59us (1111us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762685.441820:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 106 00000100:00100000:2.0:1789762685.441961:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074590336:0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.441991:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c8170e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074590848:0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.442058:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c8172d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074591360:0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.442124:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ca316680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074589824:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762685.688172:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00080000:2.0:1789762686.083219:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762686.088594:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067232128 00000100:00100000:2.0:1789762686.088608:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067232128 00000100:00100000:2.0:1789762686.088639:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 159330135371 time 159330187891 state 0x0 ref 1 00000100:00100000:2.0:1789762686.088648:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 159530187891 time 159330187891 state 0x1 ref 1 00000100:00100000:2.0:1789762686.483319:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 159730279886 time 159730390962 state 0x0 ref 1 00000100:00100000:2.0:1789762686.483347:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ca314380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067232256:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762686.483382:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174004 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762686.483396:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762686.483400:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762686.483404:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174004 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762686.483416:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174004 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762686.486318:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174004 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762686.486329:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174004 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762686.486337:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174004 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762686.486355:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762686.486360:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174004 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762686.486366:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174004 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762686.486379:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174004 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762686.486384:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174004 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762686.486431:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ca314380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067232256:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3097us (198054us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762686.486437:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 188 00000100:00100000:2.0:1789762686.486442:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 159730279886 time 159730390962 state 0x2 ref 0 00000100:00100000:2.0:1789762686.688241:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067232512 00000100:00100000:2.0:1789762686.688249:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067232512 00000100:00100000:2.0:1789762686.688282:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 159930390962 time 159930487931 state 0x0 ref 1 00000100:00100000:2.0:1789762686.688291:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 160130487931 time 159930487931 state 0x1 ref 1 00000100:00100000:2.0:1789762687.083601:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 160330577452 time 160330656267 state 0x0 ref 1 00000100:00100000:2.0:1789762687.083628:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ca315500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067232640:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762687.083677:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174019 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762687.083704:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762687.083709:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762687.083717:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174019 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762687.083732:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174019 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762687.086859:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174019 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762687.086874:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174019 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762687.086884:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174019 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762687.086892:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762687.086897:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174019 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762687.086908:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174019 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762687.086954:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174019 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762687.086960:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f174019 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762687.087008:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ca315500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067232640:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3392us (198335us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762687.087014:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 191 00000100:00100000:2.0:1789762687.087019:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 160330577452 time 160330656267 state 0x2 ref 0 00000100:00100000:2.0:1789762687.690448:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067233152 00000100:00100000:2.0:1789762687.690457:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067233152 00000100:00100000:2.0:1789762687.690490:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 160930828034 time 160930933059 state 0x0 ref 1 00000100:00100000:2.0:1789762687.690499:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 161130933059 time 160930933059 state 0x1 ref 1 00000100:00100000:2.0:1789762688.084079:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 161331047004 time 161331151277 state 0x0 ref 1 00000100:00100000:2.0:1789762688.084119:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1771180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067233280:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762688.084162:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f17403c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762688.084177:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762688.084180:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762688.084185:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f17403c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762688.084210:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f17403c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762688.086902:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f17403c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762688.086913:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f17403c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762688.086938:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f17403c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762688.086944:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762688.086948:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f17403c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762688.086963:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f17403c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762688.086979:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f17403c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762688.086985:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f17403c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762688.087045:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1771180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067233280:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2951us (198393us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762688.087053:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 196 00000100:00100000:2.0:1789762688.087057:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 161331047004 time 161331151277 state 0x2 ref 0 00010000:00010000:2.0:1789762688.640068:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762688.689633:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762689.090688:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067234048 00000100:00100000:2.0:1789762689.090698:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067234048 00000100:00100000:2.0:1789762689.090760:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 162331564357 time 162331629239 state 0x0 ref 1 00000100:00100000:2.0:1789762689.090776:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 162531629239 time 162331629239 state 0x1 ref 1 00000100:00100000:2.0:1789762689.290484:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762689.480146:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067234432 00000100:00100000:2.0:1789762689.480161:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067234432 00000100:00100000:2.0:1789762689.480260:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067234688 00000100:00100000:2.0:1789762689.480272:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067234688 00000100:00100000:2.0:1789762689.480350:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067234816 00000100:00100000:2.0:1789762689.480355:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067234816 00000100:00100000:2.0:1789762689.480433:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 141 00000100:00100000:2.0:1789762689.480456:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1748a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067234560:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762689.480516:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1748a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067234560:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 64us (281us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762689.480550:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 141 00000100:00100000:2.0:1789762689.484774:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 162731771474 time 162731849142 state 0x0 ref 1 00000100:00100000:2.0:1789762689.484788:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c8172300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067234176:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762689.484831:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f17406d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762689.484844:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762689.484848:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762689.484852:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f17406d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762689.484863:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f17406d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762689.487600:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f17406d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762689.487610:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f17406d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762689.487617:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f17406d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762689.487622:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca393400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762689.487625:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f17406d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762689.487632:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f17406d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762689.487657:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f17406d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762689.487663:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f17406d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762689.487727:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c8172300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067234176:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2941us (197246us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762689.487735:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 203 00000100:00100000:2.0:1789762689.487739:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 162731771474 time 162731849142 state 0x2 ref 0 00000100:00100000:2.0:1789762690.092385:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067235328 00000100:00100000:2.0:1789762690.092395:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067235328 00000100:00100000:2.0:1789762690.092440:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 163332082969 time 163332181381 state 0x0 ref 1 00000100:00100000:2.0:1789762690.092450:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 163532181381 time 163332181381 state 0x1 ref 1 00000100:00100000:2.0:1789762690.485395:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 163732287196 time 163732461932 state 0x0 ref 1 00000100:00100000:2.0:1789762690.485436:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7831880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067235456:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762690.485494:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f174090 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762690.485515:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762690.485519:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762690.485525:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f174090 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762690.485559:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f174090 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762690.490134:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f174090 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762690.490151:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f174090 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762690.490159:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f174090 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762690.490164:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca393400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762690.490168:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f174090 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762690.490175:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f174090 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762690.490191:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f174090 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762690.490196:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f174090 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762690.490259:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7831880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067235456:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4843us (199343us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762690.490269:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 208 00000100:00100000:2.0:1789762690.490275:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 163732287196 time 163732461932 state 0x2 ref 0 00000100:00100000:2.0:1789762690.560724:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762690.560771:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c8170000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074592256:0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.560846:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074592640 00000100:00100000:2.0:1789762690.560854:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074592640 00000100:00100000:2.0:1789762690.560953:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074593152 00000100:00100000:2.0:1789762690.560959:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074593152 00000100:00100000:2.0:1789762690.561030:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074592896 00000100:00100000:2.0:1789762690.561037:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074592896 00000100:00100000:2.0:1789762690.561070:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 143 00000100:00100000:2.0:1789762690.561089:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7830700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074592640:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.561152:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7830700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074592640:12345-0@lo:400:kworker.0 Request processed in 65us (494us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762690.561182:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 143 00000100:00100000:2.0:1789762690.561224:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 144 00000100:00100000:2.0:1789762690.561238:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7833800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074593152:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.561307:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7833800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074593152:12345-0@lo:400:kworker.0 Request processed in 69us (519us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762690.561325:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 144 00000100:00100000:2.0:1789762690.561365:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074593280 00000100:00100000:2.0:1789762690.561370:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074593280 00000100:00100000:2.0:1789762690.561403:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 145 00000100:00100000:2.0:1789762690.561429:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074592896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.561500:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074592896:12345-0@lo:400:kworker.0 Request processed in 72us (612us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762690.561542:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 145 00000100:00100000:2.0:1789762690.561591:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:2.0:1789762690.561616:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7833100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074593280:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.561728:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7833100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074593280:12345-0@lo:400:kworker.0 Request processed in 120us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762690.561744:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00080000:2.0:1789762690.561802:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762690.561812:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762690.561821:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762690.561831:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762690.561860:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 56 00000100:00100000:2.0:1789762690.561880:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7831f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074593664:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.561979:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7831f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074593664:12345-0@lo:400:kworker.0 Request processed in 101us (992us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762690.562002:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 56 00000100:00100000:2.0:1789762690.562116:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762690.562198:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c7831c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074591872:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762690.562258:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762690.562349:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074592000:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762690.562410:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c8173b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074592384:0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.562448:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c8170a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074592896:0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.562513:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074593408 00000100:00100000:2.0:1789762690.562539:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074593408 00000100:00100000:2.0:1789762690.562604:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074593536 00000100:00100000:2.0:1789762690.562610:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074593536 00000100:00100000:2.0:1789762690.562695:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 58 00000100:00100000:2.0:1789762690.562712:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e4e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074593408:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.562786:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e4e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074593408:12345-0@lo:400:kworker.0 Request processed in 84us (538us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762690.562801:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 58 00000100:00100000:2.0:1789762690.562879:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c8171880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074593408:0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.563021:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c8171180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074592768:0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.563076:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c8172a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074593280:0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.563118:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074593792:0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.563168:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c8170380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074593664:0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.691364:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067235712 00000100:00100000:2.0:1789762690.691374:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067235712 00000100:00100000:2.0:1789762690.691420:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 163932461932 time 163932551534 state 0x0 ref 1 00000100:00100000:2.0:1789762690.691429:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 164132551534 time 163932551534 state 0x1 ref 1 00000100:00100000:2.0:1789762691.085705:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 164332649837 time 164332772684 state 0x0 ref 1 00000100:00100000:2.0:1789762691.085737:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7831c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067235840:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762691.085786:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f1740a5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762691.085804:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762691.085836:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762691.085844:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f1740a5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762691.085859:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f1740a5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762691.089743:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f1740a5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762691.089757:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f1740a5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762691.089765:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f1740a5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762691.089771:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca393400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762691.089775:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f1740a5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762691.089781:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f1740a5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762691.089798:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f1740a5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762691.089803:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005efc1466/0xdf5a7ac59f1740a5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762691.089879:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7831c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067235840:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4155us (197280us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762691.089890:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 211 00000100:00100000:2.0:1789762691.089897:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 164332649837 time 164332772684 state 0x2 ref 0 00000100:00100000:2.0:1789762691.692218:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067236352 00000100:00100000:2.0:1789762691.692228:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067236352 00000100:00100000:2.0:1789762691.692274:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 164932955601 time 164933092306 state 0x0 ref 1 00000100:00100000:2.0:1789762691.692289:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 165133092306 time 164933092306 state 0x1 ref 1 00000100:00100000:2.0:1789762692.086264:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 165333213041 time 165333331556 state 0x0 ref 1 00000100:00100000:2.0:1789762692.086323:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c8170000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067236480:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762692.086378:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740c8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.086401:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762692.086407:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762692.086415:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740c8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.086434:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740c8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.090150:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740c8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.090178:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740c8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.090190:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740c8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.090200:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762692.090204:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740c8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.090213:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740c8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.090231:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740c8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.090238:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740c8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762692.090304:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c8170000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067236480:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4014us (198740us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762692.090312:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 216 00000100:00100000:2.0:1789762692.090318:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 165333213041 time 165333331556 state 0x2 ref 0 00010000:00010000:2.0:1789762692.489717:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1740d6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.489737:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1740d6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.489745:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1740d6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.489752:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762692.489756:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1740d6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.489763:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1740d6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.489781:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1740d6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762692.489787:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1740d6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762692.489844:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c8173100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067236736:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3412us (198394us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762692.489852:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 218 00000100:00100000:2.0:1789762692.489859:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 165733393059 time 165733474142 state 0x2 ref 0 00000100:00100000:2.0:1789762692.692156:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067236992 00000100:00100000:2.0:1789762692.692173:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067236992 00000100:00100000:2.0:1789762692.692215:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 165933474142 time 165933614221 state 0x0 ref 1 00000100:00100000:2.0:1789762692.692226:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 166133614221 time 165933614221 state 0x1 ref 1 00010000:00010000:2.0:1789762692.736073:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762693.086827:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 166333761098 time 166333890433 state 0x0 ref 1 00000100:00100000:2.0:1789762693.086874:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c8171c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067237120:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762693.086984:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740eb lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762693.087006:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762693.087012:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762693.087028:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740eb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762693.087045:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740eb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762693.090432:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740eb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762693.090445:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740eb lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762693.090451:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740eb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762693.090469:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762693.090473:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740eb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762693.090480:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740eb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762693.090498:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740eb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762693.090503:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f1740eb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762693.090565:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c8171c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067237120:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3714us (198340us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762693.090581:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 221 00000100:00100000:2.0:1789762693.090589:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 166333761098 time 166333890433 state 0x2 ref 0 00000100:00100000:2.0:1789762693.693089:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067237632 00000100:00100000:2.0:1789762693.693099:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067237632 00000100:00100000:2.0:1789762693.693141:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 166934049291 time 166934208241 state 0x0 ref 1 00000100:00100000:2.0:1789762693.693152:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 167134208241 time 166934208241 state 0x1 ref 1 00000100:00100000:2.0:1789762694.087383:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 167334341914 time 167334449664 state 0x0 ref 1 00000100:00100000:2.0:1789762694.087443:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1749880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067237760:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762694.087535:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000033ffcb1/0xdf5a7ac59f17410e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762694.087565:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762694.087583:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762694.087592:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000033ffcb1/0xdf5a7ac59f17410e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762694.087611:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000033ffcb1/0xdf5a7ac59f17410e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762694.493477:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762694.600031:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067238400 00000100:00100000:2.0:1789762694.600047:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067238400 00000100:00100000:2.0:1789762694.600169:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067238656 00000100:00100000:2.0:1789762694.600178:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067238656 00000100:00100000:2.0:1789762694.600253:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 149 00000100:00100000:2.0:1789762694.600271:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ca329180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2396:x1876702067238400:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762694.600329:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ca329180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2396:x1876702067238400:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 61us (362us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762694.600341:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 149 00000100:00100000:2.0:1789762694.600383:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 152 00000100:00100000:2.0:1789762694.600409:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ca328380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067238784:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762694.600466:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ca328380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067238784:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 66us (333us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762694.600483:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 152 00000100:00100000:2.0:1789762694.693384:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067238912 00000100:00100000:2.0:1789762694.693395:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067238912 00000100:00100000:2.0:1789762694.693446:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 167934641769 time 167934769824 state 0x0 ref 1 00000100:00100000:2.0:1789762694.693459:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 168134769824 time 167934769824 state 0x1 ref 1 00000100:00100000:2.0:1789762695.088005:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 168334885804 time 168335062661 state 0x0 ref 1 00000100:00100000:2.0:1789762695.088047:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ca328e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067239040:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762695.088085:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f174131 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.088099:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762695.088103:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762695.088108:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f174131 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.088119:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f174131 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.091572:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f174131 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.091584:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f174131 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.091590:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f174131 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.091612:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762695.091616:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f174131 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.091622:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f174131 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.091649:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f174131 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.091655:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f174131 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x3:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762695.091702:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ca328e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067239040:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3676us (197620us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762695.091709:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 231 00000100:00100000:2.0:1789762695.091714:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff83c000 key uid={500} rule@0xffff9c37d227c900 gen 0 rate 5 depth 3 token 0 deadline 168334885804 time 168335062661 state 0x2 ref 0 00000100:00100000:2.0:1789762695.093915:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762695.681036:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e4f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074594944:0@lo:400:kworker.0 00000100:00100000:2.0:1789762695.681132:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762695.681187:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e4d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074595456:0@lo:400:kworker.0 00000100:00100000:2.0:1789762695.681258:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762695.681314:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1789762695.681331:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e4c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074595072:0@lo:400:kworker.0 00000100:00100000:2.0:1789762695.681418:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762695.681489:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e4d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074595584:0@lo:400:kworker.0 00000100:00100000:2.0:1789762695.681578:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762695.681701:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074594944 00000100:00100000:2.0:1789762695.681711:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074594944 00000100:00100000:2.0:1789762695.681795:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074595200 00000100:00100000:2.0:1789762695.681802:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074595200 00000100:00100000:2.0:1789762695.681862:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 113 00000100:00100000:2.0:1789762695.681888:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e4c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074594944:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762695.681997:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e4c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074594944:12345-0@lo:400:kworker.0 Request processed in 116us (874us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762695.682010:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 113 00000100:00100000:2.0:1789762695.682051:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 114 00000100:00100000:2.0:1789762695.682079:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d174ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074595200:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762695.682175:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d174ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074595200:12345-0@lo:400:kworker.0 Request processed in 88us (502us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762695.682199:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 114 00000100:00100000:2.0:1789762695.682285:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:2.0:1789762695.682315:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074595584:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762695.682420:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074595584:12345-0@lo:400:kworker.0 Request processed in 117us (847us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762695.682452:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:2.0:1789762695.715171:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762695.723309:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762695.728315:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067240192 00000100:00100000:2.0:1789762695.728324:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067240192 00000100:00100000:2.0:1789762695.728378:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 94 00000100:00100000:2.0:1789762695.728385:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d174b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:11450:x1876702067240192:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:2.0:1789762695.728400:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762695.728413:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf800/0xdf5a7ac59f17415b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c39a expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762695.728481:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x4:0x0]: returning size 16384 00010000:00010000:2.0:1789762695.728569:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf800/0xdf5a7ac59f17415b lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c39a expref: 13 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.728579:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762695.728581:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762695.728584:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf800/0xdf5a7ac59f17415b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c39a expref: 13 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.728595:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8cf800, rc 0) 00000100:00100000:2.0:1789762695.728645:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d174b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:11450:x1876702067240192:12345-192.168.203.37@tcp:101:rm.0 Request processed in 260us (426us total) trans 4294967333 rc 301/301 00000100:00100000:2.0:1789762695.728651:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 94 00000100:00100000:2.0:1789762695.730103:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762695.732321:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:11450:x1876702067240320:12345-192.168.203.37@tcp:37:rm.0 Request processed in 2111us (2215us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762695.732329:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 20 00000100:00100000:2.0:1789762695.734543:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067240832 00000100:00100000:2.0:1789762695.734549:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067240832 00000100:00100000:2.0:1789762695.734581:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 95 00000100:00100000:2.0:1789762695.734587:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ca328700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:11450:x1876702067240832:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:2.0:1789762695.734606:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762695.734614:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174169 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c3a1 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.734674:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174170 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.734685:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174170 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x4:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.734693:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174170 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x4:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762695.734699:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x4:0x0]: returning size 16384 00010000:00010000:2.0:1789762695.734750:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174169 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3a1 expref: 16 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.734762:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174170 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x4:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3a1 expref: 15 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.734768:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x4:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762695.734770:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1789762695.734772:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8cf600, rc 0) 00000100:00100000:2.0:1789762695.734802:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ca328700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:11450:x1876702067240832:12345-192.168.203.37@tcp:101:rm.0 Request processed in 215us (300us total) trans 4294967335 rc 0/0 00000100:00100000:2.0:1789762695.734807:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 95 00000100:00100000:2.0:1789762695.735787:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762695.736849:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067241216 00000100:00100000:2.0:1789762695.736855:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067241216 00000100:00100000:2.0:1789762695.736892:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 97 00000100:00100000:2.0:1789762695.736898:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:11450:x1876702067241216:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:2.0:1789762695.736950:0:11692:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789762695.736968:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0c00/0xdf5a7ac59f173e7c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c385 expref: 15 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.736984:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932400/0xdf5a7ac59f173b2d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c362 expref: 14 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.736992:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6:0x0].0x0 (ffff9c37db743300) 00010000:00010000:2.0:1789762695.737000:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174154 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x4:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c393 expref: 13 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.737006:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x4:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762695.737008:0:11692:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762695.737017:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f17417e lrc: 3/0,1 mode: --/CW res: [0x200000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.737024:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f17417e lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.737029:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f17417e lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.737060:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf400/0xdf5a7ac59f174185 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x4:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.737066:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf400/0xdf5a7ac59f174185 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x4:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.737079:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf400/0xdf5a7ac59f174185 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x4:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.737397:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17418c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.737404:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17418c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.737410:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17418c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789762695.738276:0:11692:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].3, 64 off8384 00000004:00080000:2.0:1789762695.738286:0:11692:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 168985367682ns, next at 169985365485ns, rc = 0 00010000:00010000:2.0:1789762695.738334:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174193 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738344:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174193 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738352:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174193 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738360:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174193 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738367:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174193 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738374:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174193 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738398:0:11692:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762695.738409:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17418c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738413:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17418c lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738418:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17418c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738422:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8ce800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762695.738425:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17418c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738430:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17418c lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738437:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x6:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:2.0:1789762695.738440:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762695.738443:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17418c lrc: 2/0,0 mode: --/EX res: [0x200000402:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738453:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17418c lrc: 0/0,0 mode: --/EX res: [0x200000402:0x6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738465:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f17417e lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738475:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f17417e lrc: 2/0,0 mode: CW/CW res: [0x200000402:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738507:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f17417e lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738513:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cee00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762695.738519:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f17417e lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738527:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f17417e lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738538:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x4:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762695.738542:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762695.738545:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f17417e lrc: 2/0,0 mode: --/CW res: [0x200000402:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738550:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f17417e lrc: 0/0,0 mode: --/CW res: [0x200000402:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762695.738563:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000021ff116b reply state 00000000f4aa6417 transno 4294967336 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf400/0xdf5a7ac59f174185 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x4:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738570:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf400/0xdf5a7ac59f174185 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x4:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762695.738574:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x4:0x0].0x4009f790 (ffff9c37d3bee300) 00010000:00010000:2.0:1789762695.738576:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762695.738577:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762695.738613:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:11450:x1876702067241216:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1715us (1809us total) trans 4294967336 rc 0/0 00000100:00100000:2.0:1789762695.738619:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 97 00000100:00100000:2.0:1789762695.739640:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:2.0:1789762695.808070:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1789762696.646319:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 4 committed 00000001:00080000:2.0:1789762696.646345:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967337 is committed 00000001:00080000:2.0:1789762696.646372:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967338 is committed 00000004:00080000:2.0:1789762696.646382:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 3 committed 00000100:00100000:2.0:1789762696.646445:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d174a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074595968:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762696.646534:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1789762696.647286:0:3424:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37d174a300/2, rc 0, transno 4294967428 00000100:00100000:2.0:1789762696.647295:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d174a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074595968:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762697.728242:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074596096 00000100:00100000:2.0:1789762697.728251:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074596096 00000100:00100000:2.0:1789762697.728316:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 115 00000100:00100000:2.0:1789762697.728327:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d174a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074596096:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762697.728365:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762697.728424:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d174a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074596096:12345-0@lo:41:osp-pre-1-0.0 Request processed in 98us (230us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762697.728433:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 115 00000100:00100000:2.0:1789762698.752196:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074596224 00000100:00100000:2.0:1789762698.752205:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074596224 00000100:00100000:2.0:1789762698.752247:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 116 00000100:00100000:2.0:1789762698.752256:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee70ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074596224:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762698.752294:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762698.752348:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee70ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074596224:12345-0@lo:41:osp-pre-0-1.0 Request processed in 92us (191us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762698.752354:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 116 00010000:00010000:2.0:1789762698.880047:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762699.719891:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067241984 00000100:00100000:2.0:1789762699.719897:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067241984 00000100:00100000:2.0:1789762699.719976:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067242112 00000100:00100000:2.0:1789762699.719980:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067242112 00000100:00100000:2.0:1789762699.720021:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 159 00000100:00100000:2.0:1789762699.720035:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1748000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067241984:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762699.720097:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1748000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067241984:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 65us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762699.720105:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 159 00000100:00100000:2.0:1789762700.800482:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ca32ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074596608:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762700.800539:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762700.800572:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee8c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074597120:0@lo:400:kworker.0 00000100:00100000:2.0:1789762700.800626:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762700.800654:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb930e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074597632:0@lo:400:kworker.0 00000100:00100000:2.0:1789762700.800705:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762700.800741:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6095f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074598144:0@lo:400:kworker.0 00000100:00100000:2.0:1789762700.800783:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762700.800847:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ca32ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074596608:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762700.800977:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074597248 00000100:00100000:2.0:1789762700.800983:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074597248 00000100:00100000:2.0:1789762700.801021:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074597760 00000100:00100000:2.0:1789762700.801030:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074597760 00000100:00100000:2.0:1789762700.801067:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074597504 00000100:00100000:2.0:1789762700.801072:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074597504 00000100:00100000:2.0:1789762700.801101:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074596864 00000100:00100000:2.0:1789762700.801105:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074596864 00000100:00100000:2.0:1789762700.801124:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074597632 00000100:00100000:2.0:1789762700.801136:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074597632 00000100:00100000:2.0:1789762700.801168:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:2.0:1789762700.801180:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e4d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074597248:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762700.801236:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e4d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074597248:12345-0@lo:400:kworker.0 Request processed in 58us (919us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762700.801246:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:2.0:1789762700.801275:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:2.0:1789762700.801292:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee70f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074597504:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762700.801360:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee70f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074597504:12345-0@lo:400:kworker.0 Request processed in 74us (920us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762700.801369:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:2.0:1789762700.801406:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:2.0:1789762700.801422:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e4dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074597888:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762700.801465:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e4dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074597888:12345-0@lo:400:kworker.0 Request processed in 47us (695us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762700.801477:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00010000:00010000:2.0:1789762702.976065:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762703.872243:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074598656 00000100:00100000:2.0:1789762703.872258:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074598656 00000100:00100000:2.0:1789762703.872326:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 120 00000100:00100000:2.0:1789762703.872341:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee70c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074598656:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762703.872386:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762703.872485:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee70c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074598656:12345-0@lo:41:osp-pre-0-1.0 Request processed in 147us (326us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762703.872495:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 120 00000100:00100000:2.0:1789762703.872609:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074598656:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762704.840261:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067242880 00000100:00100000:2.0:1789762704.840279:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067242880 00000100:00100000:2.0:1789762704.840392:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 170 00000100:00100000:2.0:1789762704.840426:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2528a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067243136:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762704.840525:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2528a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067243136:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 100us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762704.840573:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 170 00000100:00080000:2.0:1789762704.840665:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762704.840674:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762704.840701:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067243008 00000100:00100000:2.0:1789762704.840709:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067243008 00000100:00100000:2.0:1789762704.840766:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 66 00000100:00100000:2.0:1789762704.840779:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067243008:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762704.840840:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067243008:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 63us (631us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762704.840849:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 66 00000100:00100000:2.0:1789762705.920186:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee70d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074599168:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762705.920271:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074598912 00000100:00100000:2.0:1789762705.920281:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074598912 00000100:00100000:2.0:1789762705.920353:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 74 00000100:00100000:2.0:1789762705.920367:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e4f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074598912:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762705.920409:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762705.920464:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e4f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074598912:12345-0@lo:13:osp-pre-0-0.0 Request processed in 103us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762705.920472:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 74 00000100:00100000:2.0:1789762705.920539:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762705.920584:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6b8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074599680:0@lo:400:kworker.0 00000100:00100000:2.0:1789762705.920618:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762705.920644:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c992ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074600192:0@lo:400:kworker.0 00000100:00100000:2.0:1789762705.920704:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074600320 00000100:00100000:2.0:1789762705.920709:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074600320 00000100:00100000:2.0:1789762705.920755:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074599680 00000100:00100000:2.0:1789762705.920760:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074599680 00000100:00100000:2.0:1789762705.920811:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074599424 00000100:00100000:2.0:1789762705.920817:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074599424 00000100:00100000:2.0:1789762705.920843:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074599936 00000100:00100000:2.0:1789762705.920848:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074599936 00000100:00100000:2.0:1789762705.920884:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:2.0:1789762705.920900:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6095f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074600320:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762705.920973:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6095f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074600320:12345-0@lo:400:kworker.0 Request processed in 76us (478us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762705.920997:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000100:00100000:2.0:1789762705.921532:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c992ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074600192:0@lo:400:kworker.0 00000100:00100000:2.0:1789762705.921597:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074600576:0@lo:400:kworker.0 00010000:00010000:2.0:1789762707.072020:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762707.968251:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074600704 00000100:00100000:2.0:1789762707.968261:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074600704 00000100:00100000:2.0:1789762707.968313:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 123 00000100:00100000:2.0:1789762707.968324:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfcd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074600704:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762707.968361:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762707.968419:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfcd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074600704:12345-0@lo:41:osp-pre-1-0.0 Request processed in 96us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762707.968444:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 123 00000100:00100000:2.0:1789762709.960108:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067243648 00000100:00100000:2.0:1789762709.960116:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067243648 00000100:00100000:2.0:1789762709.960197:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 179 00000100:00100000:2.0:1789762709.960214:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee8c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067243648:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762709.960259:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee8c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067243648:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 50us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762709.960272:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 179 00000100:00100000:2.0:1789762709.960296:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 181 00000100:00100000:2.0:1789762709.960305:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb930e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067243520:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762709.960342:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb930e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067243520:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 36us (235us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762709.960362:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 181 00000100:00080000:2.0:1789762709.960424:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762711.040162:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6ba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074600960:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762711.040223:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762711.040268:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074601472:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.040317:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762711.040362:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee760000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074601984:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.040399:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762711.040436:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8a8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074602496:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.040489:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762711.040537:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7fbb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074603008:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.040576:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762711.040619:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6ba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074600960:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762711.040660:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074602496 00000100:00100000:2.0:1789762711.040666:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074602496 00000100:00100000:2.0:1789762711.040708:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:2.0:1789762711.040720:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074602368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.040765:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074602368:12345-0@lo:400:kworker.0 Request processed in 48us (389us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762711.040773:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00080000:2.0:1789762711.040811:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762711.040817:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762711.040823:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762711.040829:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762711.040849:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762711.040888:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074601600:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.040969:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762711.041002:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff69b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074602112:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.041049:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762711.041078:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8abb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074602624:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.041130:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762711.041183:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074601088:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762711.041222:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074601600:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.041244:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff69b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074602112:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.041268:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762711.041277:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8ab100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074602752:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.041316:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762711.041348:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8abb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074602624:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.041395:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074601216 00000100:00100000:2.0:1789762711.041400:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074601216 00000100:00100000:2.0:1789762711.041439:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 81 00000100:00100000:2.0:1789762711.041452:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074601216:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762711.041469:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762711.041544:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074601216:12345-0@lo:13:osp-pre-1-0.0 Request processed in 92us (275us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762711.041561:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 81 00000100:00100000:2.0:1789762711.041600:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8ab100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074602752:0@lo:400:kworker.0 00000100:00080000:2.0:1789762711.041623:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762711.041648:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074601216:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762711.041656:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e4d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074601728:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.041662:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074602240:0@lo:400:kworker.0 00010000:00010000:2.0:1789762711.168031:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762715.080056:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762715.080085:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067244544 00000100:00100000:2.0:1789762715.080091:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067244544 00000100:00100000:2.0:1789762715.080136:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 74 00000100:00100000:2.0:1789762715.080152:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067244544:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762715.080215:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067244544:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 65us (210us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762715.080239:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 74 00000100:00080000:2.0:1789762715.080276:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762715.080281:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762715.080288:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:2.0:1789762715.264041:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762716.160437:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762716.160446:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762716.160488:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074603904 00000100:00100000:2.0:1789762716.160495:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074603904 00000100:00100000:2.0:1789762716.160542:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 129 00000100:00100000:2.0:1789762716.160556:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074603904:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.160621:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074603904:12345-0@lo:400:kworker.0 Request processed in 69us (368us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762716.160632:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 129 00000100:00100000:2.0:1789762716.160666:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074603776:0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.160717:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762716.160754:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b96300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074604288:0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.160784:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762716.160808:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e4d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074604800:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762716.160844:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762716.160870:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b94380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074605312:0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.160939:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762716.160994:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074604288 00000100:00100000:2.0:1789762716.161000:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074604288 00000100:00100000:2.0:1789762716.161044:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 130 00000100:00100000:2.0:1789762716.161059:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1771f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074604288:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.161113:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1771f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074604288:12345-0@lo:400:kworker.0 Request processed in 57us (331us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762716.161138:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 130 00000100:00080000:2.0:1789762716.161175:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762716.161181:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762716.161186:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762716.161221:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b94700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074603776:0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.161235:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b96300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074604288:0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.161250:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b94380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074605312:0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.161305:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074605184 00000100:00100000:2.0:1789762716.161313:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074605184 00000100:00100000:2.0:1789762716.161355:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 77 00000100:00100000:2.0:1789762716.161368:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074605184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.161434:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074605184:12345-0@lo:400:kworker.0 Request processed in 68us (187us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762716.161452:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 77 00000100:00100000:2.0:1789762719.232374:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074605568 00000100:00100000:2.0:1789762719.232407:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074605568 00000100:00100000:2.0:1789762719.232525:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 132 00000100:00100000:2.0:1789762719.232542:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074605568:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762719.232593:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762719.232674:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074605568:12345-0@lo:41:osp-pre-0-1.0 Request processed in 137us (387us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762719.232685:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 132 00010000:00010000:2.0:1789762719.360060:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762720.200632:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067245184 00000100:00100000:2.0:1789762720.200647:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067245184 00000100:00100000:2.0:1789762720.200772:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 199 00000100:00100000:2.0:1789762720.200812:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b94a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067245184:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762720.200891:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b94a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067245184:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 90us (414us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762720.200906:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 199 00000100:00100000:2.0:1789762720.205086:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067245312 00000100:00100000:2.0:1789762720.205098:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067245312 00000100:00100000:2.0:1789762720.205221:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 64 00000100:00100000:2.0:1789762720.205254:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2394:x1876702067245312:12345-192.168.203.37@tcp:9:kworker.0 00000020:00000080:2.0:1789762720.205317:0:9038:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xdf5a7ac59f17369c 00000100:00080000:2.0:1789762720.205341:0:9038:0:(import.c:65:import_set_state_nolock()) ffff9c37ff6d4800 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1789762720.205416:0:9038:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294967428 00000100:00100000:2.0:1789762720.205884:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+2:2394:x1876702067245312:12345-192.168.203.37@tcp:9:kworker.0 Request processed in 640us (891us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762720.205902:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 64 00000100:00100000:2.0:1789762721.280683:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074605824 00000100:00100000:2.0:1789762721.280695:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074605824 00000100:00100000:2.0:1789762721.280758:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074606336 00000100:00100000:2.0:1789762721.280764:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074606336 00000100:00100000:2.0:1789762721.280794:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074605952 00000100:00100000:2.0:1789762721.280798:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074605952 00000100:00100000:2.0:1789762721.280822:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074606080 00000100:00100000:2.0:1789762721.280827:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074606080 00000100:00100000:2.0:1789762721.280858:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 86 00000100:00100000:2.0:1789762721.280879:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074605824:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789762721.280970:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762721.281067:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074605824:12345-0@lo:13:osp-pre-0-1.0 Request processed in 190us (853us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.281114:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 86 00000100:00100000:2.0:1789762721.281158:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 89 00000100:00100000:2.0:1789762721.281178:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074606080:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789762721.281200:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762721.281267:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074606080:12345-0@lo:13:osp-pre-1-1.0 Request processed in 92us (770us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.281298:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 89 00000100:00080000:2.0:1789762721.281417:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762721.281428:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762721.281588:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074606592 00000100:00100000:2.0:1789762721.281595:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074606592 00000100:00100000:2.0:1789762721.281638:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074607232 00000100:00100000:2.0:1789762721.281645:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074607232 00000100:00100000:2.0:1789762721.281708:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:2.0:1789762721.281729:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a36a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074606208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762721.281814:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a36a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074606208:12345-0@lo:400:kworker.0 Request processed in 88us (1166us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.281832:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:2.0:1789762721.281916:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762721.281997:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bc6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074607616:0@lo:400:kworker.0 00000100:00100000:2.0:1789762721.282049:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762721.282123:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b8ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074606080:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762721.282160:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f63ae680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074606592:0@lo:400:kworker.0 00000100:00100000:2.0:1789762721.282198:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bc4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074607104:0@lo:400:kworker.0 00000100:00100000:2.0:1789762721.282321:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bc6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074607616:0@lo:400:kworker.0 00000100:00100000:2.0:1789762721.681773:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 100 00000100:00100000:2.0:1789762721.681817:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067245824:12345-192.168.203.37@tcp:41:mkdir.0 00000004:00000024:2.0:1789762721.681859:0:13389:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762721.681997:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067245824:12345-192.168.203.37@tcp:41:mkdir.0 Request processed in 208us (427us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.682006:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 100 00000100:00100000:2.0:1789762721.682062:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074607744 00000100:00100000:2.0:1789762721.682068:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074607744 00000100:00100000:2.0:1789762721.682122:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1789762721.682138:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074607744:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1789762721.682242:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074607744:12345-0@lo:1000:mdt00_002.0 Request processed in 109us (213us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.682249:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1789762721.682489:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074607872 00000100:00100000:2.0:1789762721.682498:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074607872 00000100:00100000:2.0:1789762721.682548:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 136 00000100:00100000:2.0:1789762721.682559:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e4d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074607872:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1789762721.682945:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e4d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074607872:12345-0@lo:1000:mdt00_002.0 Request processed in 387us (526us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789762721.682954:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 136 00000100:00100000:2.0:1789762721.684771:0:6423:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b96300 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6423:1876702074608128:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762721.684799:0:6423:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1741cb lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1741c4 expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.684826:0:6423:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762721.684836:0:6423:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762721.684841:0:6423:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1741cb lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1741c4 expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.684850:0:6423:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5933a00, rc 0) 00000100:00100000:2.0:1789762721.684894:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6420:x1876702074608000:12345-0@lo:101:mdt00_002.0 Request processed in 1543us (1738us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.684900:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 137 00000100:00100000:2.0:1789762721.685450:0:8068:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074608256 00000100:00100000:2.0:1789762721.685459:0:8068:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074608256 00000100:00100000:2.0:1789762721.685518:0:8068:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 37 00000100:00100000:2.0:1789762721.685533:0:8068:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:6404:x1876702074608256:12345-0@lo:105: 00010000:00010000:2.0:1789762721.685546:0:8068:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c0031c00/0xdf5a7ac59f1741c4 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1741cb expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.685559:0:8068:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c0031c00/0xdf5a7ac59f1741c4 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1741cb expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.685572:0:8068:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c0031c00/0xdf5a7ac59f1741c4 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1741cb expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.685584:0:8068:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c0031c00/0xdf5a7ac59f1741c4 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f1741cb expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.685595:0:8068:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c0031c00/0xdf5a7ac59f1741c4 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f1741cb expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.685607:0:8068:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c0031c00/0xdf5a7ac59f1741c4 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f1741cb expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.685653:0:8068:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c37c0031c00) 00000100:00100000:2.0:1789762721.685719:0:8068:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:6404:x1876702074608256:12345-0@lo:105: Request processed in 188us (316us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.685732:0:8068:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 37 00000100:00100000:2.0:1789762721.686155:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074608384:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1789762721.686187:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c0030c00/0xdf5a7ac59f1741d2 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1741d9 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.686223:0:6420:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c0030c00/0xdf5a7ac59f1741d2 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1741d9 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762721.686243:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1789762721.686298:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b8ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074608512:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1789762721.686376:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762721.686410:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d1412600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762721.705603:0:6403:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074609664 00000100:00100000:2.0:1789762721.705616:0:6403:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074609664 00000100:00100000:2.0:1789762721.705700:0:6403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 38 00000100:00100000:2.0:1789762721.705714:0:6403:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:11692:x1876702074609664:12345-0@lo:104: 00010000:00010000:2.0:1789762721.705731:0:6403:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c0030c00/0xdf5a7ac59f1741d2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f1741d9 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.705857:0:6403:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1789762721.705888:0:6403:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:11692:x1876702074609664:12345-0@lo:104: Request processed in 157us (346us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.705919:0:6403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 38 00000001:00080000:2.0:1789762721.724290:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967339 is committed 00000100:00100000:2.0:1789762721.727683:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c94a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074609792:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789762721.730269:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762721.732186:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067246976 00000100:00100000:2.0:1789762721.732197:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067246976 00000100:00100000:2.0:1789762721.732258:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 105 00000100:00100000:2.0:1789762721.732277:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:12077:x1876702067246976:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762721.732307:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762721.732324:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f174218 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c3c4 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.732398:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cec00/0xdf5a7ac59f17421f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.732416:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cec00/0xdf5a7ac59f17421f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.732451:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cec00/0xdf5a7ac59f17421f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762721.732487:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762721.732493:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762721.732518:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cec00/0xdf5a7ac59f17421f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.732554:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f174218 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3c4 expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.732583:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cec00/0xdf5a7ac59f17421f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3c4 expref: 7 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.732618:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762721.732624:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762721.732629:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8cec00, rc 0) 00000100:00100000:2.0:1789762721.732713:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:12077:x1876702067246976:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 423us (579us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.732725:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 105 00000100:00100000:2.0:1789762721.737399:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762721.737492:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762721.748171:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067248256 00000100:00100000:2.0:1789762721.748178:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067248256 00000100:00100000:2.0:1789762721.748217:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 109 00000100:00100000:2.0:1789762721.748226:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f63ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:12078:x1876702067248256:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:2.0:1789762721.748239:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762721.748253:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f174257 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c3d9 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.748305:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfc00/0xdf5a7ac59f17425e lrc: 3/1,0 mode: --/PR res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.748321:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfc00/0xdf5a7ac59f17425e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.748334:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfc00/0xdf5a7ac59f17425e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762721.748356:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x6:0x0]: returning size 16384 00000004:00200000:2.0:1789762721.748359:0:11692:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762721.748381:0:11692:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfc00/0xdf5a7ac59f17425e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.748403:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f174257 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3d9 expref: 13 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.748420:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfc00/0xdf5a7ac59f17425e lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3d9 expref: 12 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.748442:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:2.0:1789762721.748448:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762721.748452:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8cfc00, rc 0) 00000100:00100000:2.0:1789762721.748519:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f63ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:12078:x1876702067248256:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 292us (397us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.748532:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 109 00000100:00100000:2.0:1789762721.749821:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067248512 00000100:00100000:2.0:1789762721.749827:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067248512 00000100:00100000:2.0:1789762721.749864:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 110 00000100:00100000:2.0:1789762721.749870:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:12078:x1876702067248512:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:2.0:1789762721.749896:0:13389:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762721.749911:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfc00/0xdf5a7ac59f17425e lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3d9 expref: 12 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.749953:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:2.0:1789762721.749957:0:13389:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762721.749968:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174265 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.749977:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174265 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.749992:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174265 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762721.750169:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fbe8b6cd reply state 00000000ae15b898 transno 4294967331 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174265 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.750181:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174265 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.750188:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:2.0:1789762721.750190:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:2.0:1789762721.750192:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762721.750198:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x6:0x0]: returning size 16384 00000100:00100000:2.0:1789762721.750249:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:12078:x1876702067248512:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 377us (474us total) trans 4294967331 rc 0/0 00000100:00100000:2.0:1789762721.750256:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 110 00000100:00100000:2.0:1789762721.778287:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762721.780262:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 66 00000100:00100000:2.0:1789762721.780283:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0+-99:2394:x1876702067249920:12345-192.168.203.37@tcp:8:touch.500 00010000:00080000:2.0:1789762721.780315:0:9037:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp t0 exp 0000000000000000 cur 195 last 0 00000020:00000080:2.0:1789762721.780358:0:9037:0:(genops.c:1385:class_connect()) connect: client fa6fc127-0d58-47ca-82d1-2c8ad7b086c1, cookie 0xdf5a7ac59f1742ce 00002000:00100000:2.0:1789762721.780374:0:9037:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli fa6fc127-0d58-47ca-82d1-2c8ad7b086c1/ffff9c36cbb24000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:2.0:1789762721.780390:0:9037:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:2.0:1789762721.780693:0:9037:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:2.0:1789762721.780716:0:9037:0:(import.c:65:import_set_state_nolock()) ffff9c36cbb26000 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789762721.780752:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2394:x1876702067249920:12345-192.168.203.37@tcp:8:touch.500 Request processed in 470us (657us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.780770:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 66 00000100:00100000:2.0:1789762721.781866:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067249792 00000100:00100000:2.0:1789762721.781891:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067249792 00000100:00100000:2.0:1789762721.781997:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 67 00000100:00100000:2.0:1789762721.782018:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:0:x1876702067249792:12345-192.168.203.37@tcp:2:touch.500 00000100:00100000:2.0:1789762721.800257:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:0:x1876702067250048:12345-192.168.203.37@tcp:2:touch.500 Request processed in 18207us (18411us total) trans 4294967323 rc 0/0 00000100:00100000:2.0:1789762721.800269:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 68 00000100:00100000:2.0:1789762721.826853:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067250560 00000100:00100000:2.0:1789762721.826863:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067250560 00000100:00100000:2.0:1789762721.826970:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 116 00000100:00100000:2.0:1789762721.826984:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:12086:x1876702067250560:12345-192.168.203.37@tcp:101:touch.500 00010000:00010000:2.0:1789762721.827012:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762721.827037:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf000/0xdf5a7ac59f1742d5 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c3fc expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827186:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1742dc lrc: 3/0,1 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827200:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1742dc lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827210:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1742dc lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827284:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1742e3 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827308:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1742e3 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827315:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1742e3 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762721.827694:0:13389:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x5:0x0], index 1: 0 00000004:00080000:2.0:1789762721.827700:0:13389:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x5:0x0], index 0: 0 00010000:00010000:2.0:1789762721.827866:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1742ea lrc: 3/1,0 mode: --/PR res: [0x240000402:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827892:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1742ea lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827899:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1742ea lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827927:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1742dc lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827933:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1742dc lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827956:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1742dc lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827962:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8ce000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762721.827966:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1742dc lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.827973:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1742dc lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.828028:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:2.0:1789762721.828034:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762721.828041:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1742dc lrc: 2/0,0 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.828053:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1742dc lrc: 0/0,0 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762721.828072:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000010419bc7 reply state 00000000bc46d006 transno 4294967335 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1742e3 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.828086:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1742e3 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.828093:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x2b0e463d (ffff9c37d3beed00) 00010000:00010000:2.0:1789762721.828095:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762721.828098:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762721.828147:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf000/0xdf5a7ac59f1742d5 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3fc expref: 23 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.828160:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1742ea lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3fc expref: 22 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.828168:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x8:0x0].0x0 (ffff9c37d3beef00) 00010000:00010000:2.0:1789762721.828170:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762721.828172:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8ce600, rc 0) 00000100:00100000:2.0:1789762721.828242:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:12086:x1876702067250560:12345-192.168.203.37@tcp:101:touch.500 Request processed in 1262us (1469us total) trans 4294967335 rc 0/0 00000100:00100000:2.0:1789762721.828248:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 116 00000100:00100000:2.0:1789762721.830153:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762721.833351:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067251072 00000100:00100000:2.0:1789762721.833358:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067251072 00000100:00100000:2.0:1789762721.833392:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067251200 00000100:00100000:2.0:1789762721.833395:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067251200 00000100:00100000:2.0:1789762721.833447:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 69 00000100:00100000:2.0:1789762721.833462:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067251072:12345-192.168.203.37@tcp:2:touch.500 00000100:00100000:2.0:1789762721.833791:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067251072:12345-192.168.203.37@tcp:2:touch.500 Request processed in 333us (488us total) trans 4294967324 rc 0/0 00000100:00100000:2.0:1789762721.833803:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 69 00000100:00100000:2.0:1789762721.834754:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762721.859804:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762721.862518:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b8ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:12090:x1876702067251968:12345-192.168.203.37@tcp:37:bash.0 Request processed in 2585us (2712us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.862527:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 26 00000100:00100000:2.0:1789762721.864966:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762721.871148:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067252352 00000100:00100000:2.0:1789762721.871155:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067252352 00000100:00100000:2.0:1789762721.871194:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 121 00000100:00100000:2.0:1789762721.871202:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:12090:x1876702067252352:12345-192.168.203.37@tcp:101:chmod.500 00010000:00010000:2.0:1789762721.871215:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762721.871226:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf800/0xdf5a7ac59f174322 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c418 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871263:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174329 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871273:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174329 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871290:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174329 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871299:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174330 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871305:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174330 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871311:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174330 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871368:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174337 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871374:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174337 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871380:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174337 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762721.871399:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x7:0x0]: returning size 0 00000004:00200000:2.0:1789762721.871402:0:11692:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762721.871447:0:11692:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174337 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871457:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174337 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871467:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174329 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871490:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174329 lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871499:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174329 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871507:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8ce200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762721.871512:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174329 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871523:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174329 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871537:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:2.0:1789762721.871540:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762721.871545:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174329 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871551:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f174329 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871563:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174330 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871568:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174330 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871573:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174330 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871577:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cf600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762721.871585:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174330 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871591:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174330 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871597:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x73550ecb (ffff9c37c72ba300) 00010000:00010000:2.0:1789762721.871599:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762721.871602:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174330 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871606:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f174330 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871628:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf800/0xdf5a7ac59f174322 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c418 expref: 31 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871638:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174337 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c418 expref: 30 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.871645:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7:0x0].0x0 (ffff9c37db743300) 00010000:00010000:2.0:1789762721.871647:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762721.871649:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8cf200, rc 0) 00000100:00100000:2.0:1789762721.871686:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:12090:x1876702067252352:12345-192.168.203.37@tcp:101:chmod.500 Request processed in 485us (634us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.871692:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 121 00000100:00100000:2.0:1789762721.873124:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067252480 00000100:00100000:2.0:1789762721.873132:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067252480 00000100:00100000:2.0:1789762721.873170:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067252608 00000100:00100000:2.0:1789762721.873173:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067252608 00000100:00100000:2.0:1789762721.873234:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 72 00000100:00100000:2.0:1789762721.873253:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067252608:12345-192.168.203.37@tcp:101:chmod.500 00010000:00010000:2.0:1789762721.873264:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762721.873282:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174345 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c426 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762721.873308:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x4:0x0] initial LVB size: 0, mtime: 0x6aad9ca1, atime: 0x6aad9ca1, ctime: 0x6aad9ca1, blocks: 0x1 00010000:00010000:2.0:1789762721.873323:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4:0x0].0x0 since=195s v=(0 0) 00010000:00010000:2.0:1789762721.873332:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762721.873341:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174345 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c426 expref: 7 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.873356:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174345 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c426 expref: 7 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.873365:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8cee00, rc 0) 00000100:00100000:2.0:1789762721.873397:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067252608:12345-192.168.203.37@tcp:101:chmod.500 Request processed in 148us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.873405:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 72 00000100:00100000:2.0:1789762721.874787:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067252864 00000100:00100000:2.0:1789762721.874794:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067252864 00000100:00100000:2.0:1789762721.874833:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 122 00000100:00100000:2.0:1789762721.874841:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:12090:x1876702067252864:12345-192.168.203.37@tcp:36:chmod.500 00010000:00010000:2.0:1789762721.874868:0:13389:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762721.874882:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174337 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c418 expref: 30 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.874907:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7:0x0].0x0 (ffff9c37db743300) 00010000:00010000:2.0:1789762721.874911:0:13389:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762721.874949:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17434c lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.874958:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17434c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.874965:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17434c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762721.875090:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c7a488f2 reply state 0000000063d30bc7 transno 4294967340 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17434c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.875100:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17434c lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.875106:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7:0x0].0x0 (ffff9c37db743300) 00010000:00010000:2.0:1789762721.875109:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:2.0:1789762721.875111:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762721.875165:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:12090:x1876702067252864:12345-192.168.203.37@tcp:36:chmod.500 Request processed in 324us (427us total) trans 4294967340 rc 0/0 00000100:00100000:2.0:1789762721.875171:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 122 00000100:00100000:2.0:1789762721.876227:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067253120 00000100:00100000:2.0:1789762721.876235:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067253120 00000100:00100000:2.0:1789762721.876297:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 74 00000100:00100000:2.0:1789762721.876313:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067253120:12345-192.168.203.37@tcp:2:chmod.500 00002000:00010000:3.0:1789762721.876492:0:8052:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1789762721 -> 1789762721 00002000:00010000:3.0:1789762721.876500:0:8052:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 1 -> 1 00002000:00010000:2.0:1789762721.876537:0:9037:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00000100:00100000:3.0:1789762721.876538:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067252992:12345-192.168.203.37@tcp:2:chmod.500 Request processed in 271us (400us total) trans 4294967431 rc 0/0 00000100:00100000:3.0:1789762721.876546:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 73 00002000:00010000:2.0:1789762721.876554:0:9037:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb mtime from disk: 1789762721 -> 1789762721 00002000:00010000:2.0:1789762721.876558:0:9037:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb atime from disk: 1789762721 -> 1789762721 00002000:00010000:2.0:1789762721.876560:0:9037:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb ctime from disk: 1789762721 -> 1789762721 00002000:00010000:2.0:1789762721.876563:0:9037:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1789762721.876623:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067253120:12345-192.168.203.37@tcp:2:chmod.500 Request processed in 315us (452us total) trans 4294967325 rc 0/0 00000100:00100000:2.0:1789762721.876633:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 74 00000100:00100000:3.0:1789762721.878035:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067253248 00000100:00100000:3.0:1789762721.878042:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067253248 00000100:00100000:3.0:1789762721.878085:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 123 00000100:00100000:3.0:1789762721.878097:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:12090:x1876702067253248:12345-192.168.203.37@tcp:101:chmod.500 00010000:00010000:3.0:1789762721.878112:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762721.878126:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174353 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c42d expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878172:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f17435a lrc: 3/1,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878186:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f17435a lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878198:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f17435a lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878215:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174361 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878226:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174361 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878236:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174361 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878300:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174368 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878321:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174368 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878331:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174368 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762721.878357:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x8:0x0]: returning size 0 00000004:00200000:3.0:1789762721.878361:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762721.878378:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174368 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878390:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174368 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878400:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f17435a lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878409:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f17435a lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878418:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f17435a lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878426:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762721.878447:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f17435a lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878457:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f17435a lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878469:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:3.0:1789762721.878473:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762721.878485:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f17435a lrc: 2/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878494:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f17435a lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878509:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174361 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878519:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174361 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878527:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174361 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878535:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762721.878540:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174361 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878549:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174361 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878561:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x2b0e463d (ffff9c37d3beed00) 00010000:00010000:3.0:1789762721.878582:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762721.878587:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174361 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878596:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174361 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878631:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174353 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c42d expref: 33 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878672:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174368 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c42d expref: 32 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.878684:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x8:0x0].0x0 (ffff9c37c72ba900) 00010000:00010000:3.0:1789762721.878688:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762721.878692:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff934e00, rc 0) 00000100:00100000:3.0:1789762721.878740:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:12090:x1876702067253248:12345-192.168.203.37@tcp:101:chmod.500 Request processed in 644us (775us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762721.878752:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 123 00000100:00100000:2.0:1789762721.880199:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067253376 00000100:00100000:2.0:1789762721.880207:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067253376 00000100:00100000:2.0:1789762721.880244:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067253504 00000100:00100000:2.0:1789762721.880246:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067253504 00000100:00100000:2.0:1789762721.880283:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 76 00000100:00100000:2.0:1789762721.880301:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067253504:12345-192.168.203.37@tcp:101:chmod.500 00010000:00010000:2.0:1789762721.880320:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762721.880337:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174376 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c43b expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762721.880366:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x5:0x0] initial LVB size: 0, mtime: 0x6aad9ca1, atime: 0x6aad9ca1, ctime: 0x6aad9ca1, blocks: 0x1 00010000:00010000:2.0:1789762721.880382:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=195s v=(0 0) 00010000:00010000:2.0:1789762721.880389:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762721.880405:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174376 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c43b expref: 9 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.880449:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174376 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c43b expref: 9 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.880461:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8cf800, rc 0) 00000100:00100000:2.0:1789762721.880517:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2397:x1876702067253504:12345-192.168.203.37@tcp:101:chmod.500 Request processed in 218us (323us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.880530:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 76 00000100:00100000:3.0:1789762721.882050:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067253760 00000100:00100000:3.0:1789762721.882059:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067253760 00000100:00100000:3.0:1789762721.882103:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 124 00000100:00100000:3.0:1789762721.882113:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e4e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:12090:x1876702067253760:12345-192.168.203.37@tcp:36:chmod.500 00010000:00010000:3.0:1789762721.882161:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762721.882193:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174368 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c42d expref: 32 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.882209:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x8:0x0].0x0 (ffff9c37c72ba900) 00010000:00010000:3.0:1789762721.882212:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762721.882226:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935600/0xdf5a7ac59f17437d lrc: 3/0,1 mode: --/PW res: [0x240000402:0x8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.882237:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935600/0xdf5a7ac59f17437d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.882245:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935600/0xdf5a7ac59f17437d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762721.882356:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004254bf57 reply state 00000000e6865324 transno 4294967341 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935600/0xdf5a7ac59f17437d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.882374:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935600/0xdf5a7ac59f17437d lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.882385:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x8:0x0].0x0 (ffff9c37c72ba900) 00010000:00010000:3.0:1789762721.882388:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:3.0:1789762721.882392:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762721.882481:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e4e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:12090:x1876702067253760:12345-192.168.203.37@tcp:36:chmod.500 Request processed in 367us (485us total) trans 4294967341 rc 0/0 00000100:00100000:3.0:1789762721.882493:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 124 00000100:00100000:2.0:1789762721.883622:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067253888 00000100:00100000:2.0:1789762721.883630:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067253888 00000100:00100000:2.0:1789762721.883676:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 77 00000100:00100000:2.0:1789762721.883688:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2398:x1876702067253888:12345-192.168.203.37@tcp:2:chmod.500 00002000:00010000:2.0:1789762721.883821:0:9037:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0x5:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1789762721.883828:0:9037:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0x5:0x0] updating lvb mtime from disk: 1789762721 -> 1789762721 00002000:00010000:2.0:1789762721.883832:0:9037:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x2c0000400:0x5:0x0] updating lvb atime from disk: 1789762721 -> 1789762721 00002000:00010000:2.0:1789762721.883835:0:9037:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0x5:0x0] updating lvb ctime from disk: 1789762721 -> 1789762721 00002000:00010000:2.0:1789762721.883838:0:9037:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000400:0x5:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:3.0:1789762721.883848:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067254016 00000100:00100000:3.0:1789762721.883856:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067254016 00000100:00100000:2.0:1789762721.883897:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2398:x1876702067253888:12345-192.168.203.37@tcp:2:chmod.500 Request processed in 207us (335us total) trans 4294967326 rc 0/0 00000100:00100000:3.0:1789762721.883910:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 78 00000100:00100000:3.0:1789762721.883960:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2398:x1876702067254016:12345-192.168.203.37@tcp:2:chmod.500 00000100:00100000:2.0:1789762721.883962:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 77 00002000:00010000:3.0:1789762721.884151:0:8052:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1789762721.884156:0:8052:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb mtime from disk: 1789762721 -> 1789762721 00002000:00010000:3.0:1789762721.884159:0:8052:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb atime from disk: 1789762721 -> 1789762721 00002000:00010000:3.0:1789762721.884161:0:8052:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb ctime from disk: 1789762721 -> 1789762721 00002000:00010000:3.0:1789762721.884164:0:8052:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:3.0:1789762721.884223:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2398:x1876702067254016:12345-192.168.203.37@tcp:2:chmod.500 Request processed in 271us (418us total) trans 4294967432 rc 0/0 00000100:00100000:3.0:1789762721.884234:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 78 00000100:00100000:3.0:1789762721.909102:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067254272 00000100:00100000:3.0:1789762721.909114:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067254272 00000100:00100000:3.0:1789762721.909169:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 125 00000100:00100000:3.0:1789762721.909193:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee70f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:12094:x1876702067254272:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762721.909217:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762721.909221:0:6420:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1789762721.909243:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f17431b lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c411 expref: 33 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909263:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:3.0:1789762721.909268:0:6420:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762721.909280:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174384 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c442 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909406:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17438b lrc: 3/1,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909420:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17438b lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909454:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17438b lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909466:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174392 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909475:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174392 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909481:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174392 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909541:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f174399 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909558:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f174399 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909565:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f174399 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762721.909575:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x7:0x0]: returning size 0 00010000:00010000:3.0:1789762721.909659:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17438b lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909667:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17438b lrc: 4/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909673:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17438b lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909679:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17438b lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909685:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762721.909689:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17438b lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909696:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17438b lrc: 5/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909705:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:3.0:1789762721.909708:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762721.909712:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17438b lrc: 3/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909718:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17438b lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909737:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174392 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909743:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174392 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909749:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174392 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909753:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762721.909756:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174392 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909762:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174392 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909768:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x73550ecb (ffff9c37c72ba300) 00010000:00010000:3.0:1789762721.909771:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762721.909773:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174392 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909778:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174392 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909809:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174384 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c442 expref: 35 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909820:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934a00/0xdf5a7ac59f174399 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c442 expref: 34 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762721.909846:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7:0x0].0x0 (ffff9c37db743300) 00010000:00010000:3.0:1789762721.909850:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762721.909854:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff934a00, rc 0) 00000100:00100000:3.0:1789762721.909966:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee70f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:12094:x1876702067254272:12345-192.168.203.37@tcp:101:dd.500 Request processed in 772us (948us total) trans 4294967342 rc 0/0 00000100:00100000:3.0:1789762721.909977:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 125 00000100:00100000:2.0:1789762721.913380:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067254656 00000100:00100000:2.0:1789762721.913388:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067254656 00000100:00100000:2.0:1789762721.913452:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 79 00000100:00100000:2.0:1789762721.913469:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee8c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:12094:x1876702067254656:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:2.0:1789762721.913484:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762721.913501:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000628cf1d4/0xdf5a7ac59f1743ae lrc: 2/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c449 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.913535:0:8053:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17433e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c41f expref: 11 pid: 9038 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.913565:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762721.913569:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762721.913576:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000628cf1d4/0xdf5a7ac59f1743ae lrc: 4/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000080000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c449 expref: 11 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.913630:0:8053:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17433e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c41f expref: 11 pid: 9038 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.913646:0:8053:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17433e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c41f expref: 11 pid: 9038 timeout: 295 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762721.913663:0:8053:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b8c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074610176:192.168.203.37@tcp:104: 00000100:00100000:2.0:1789762721.913771:0:8053:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7fbee40 going to sleep for 11 seconds 00000100:00100000:2.0:1789762721.914896:0:8053:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b8c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074610176:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762721.914979:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000628cf1d4/0xdf5a7ac59f1743ae lrc: 3/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c449 expref: 11 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.915003:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8cf200, rc 0) 00000100:00100000:2.0:1789762721.915054:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee8c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:12094:x1876702067254656:12345-192.168.203.37@tcp:101:dd.500 Request processed in 1592us (1756us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.915063:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 79 00000100:00100000:2.0:1789762721.915502:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067254784 00000100:00100000:2.0:1789762721.915510:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067254784 00010000:00010000:2.0:1789762721.915549:0:8070:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17433e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c41f expref: 11 pid: 9038 timeout: 295 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762721.915584:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 53 00000100:00100000:2.0:1789762721.915594:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee8c1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:2398:x1876702067254784:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762721.915607:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762721.915613:0:8070:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17433e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c41f expref: 11 pid: 9038 timeout: 295 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.915626:0:8070:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17433e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c41f expref: 11 pid: 9038 timeout: 295 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.915642:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17433e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c41f expref: 11 pid: 9038 timeout: 295 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.915654:0:8070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762721.915657:0:8070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762721.915663:0:8070:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000628cf1d4/0xdf5a7ac59f1743ae lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c449 expref: 10 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.915672:0:8070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000628cf1d4/0xdf5a7ac59f1743ae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c449 expref: 10 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.915694:0:8070:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000628cf1d4/0xdf5a7ac59f1743ae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c449 expref: 10 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762721.915719:0:8070:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b8c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8070:1876702074610304:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762721.915767:0:8070:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7fbee40 going to sleep for 11 seconds 00000100:00100000:2.0:1789762721.917137:0:8070:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b8c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8070:1876702074610304:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762721.917168:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762721.917205:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee8c1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2398:x1876702067254784:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 1613us (1809us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.917213:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 53 00000100:00100000:2.0:1789762721.919387:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067255040 00000100:00100000:2.0:1789762721.919397:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067255040 00010000:00010000:2.0:1789762721.919423:0:6404:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174345 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c426 expref: 11 pid: 8053 timeout: 295 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762721.919495:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 54 00000100:00100000:2.0:1789762721.919505:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c96050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:2397:x1876702067255040:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762721.919517:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762721.919523:0:6404:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174345 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c426 expref: 11 pid: 8053 timeout: 295 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.919537:0:6404:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174345 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c426 expref: 11 pid: 8053 timeout: 295 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.919553:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174345 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c426 expref: 11 pid: 8053 timeout: 295 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.919601:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762721.919607:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762721.919614:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ded696a4/0xdf5a7ac59f1743b5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c450 expref: 10 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.919626:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ded696a4/0xdf5a7ac59f1743b5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c450 expref: 10 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762721.919656:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ded696a4/0xdf5a7ac59f1743b5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c450 expref: 10 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762721.919671:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b8c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074610560:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762721.919789:0:6404:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7fbee40 going to sleep for 11 seconds 00000100:00100000:3.0:1789762721.920901:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b8c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074610560:192.168.203.37@tcp:105: 00010000:00010000:3.0:1789762721.921119:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762721.921248:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c96050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2397:x1876702067255040:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 1738us (1927us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762721.921268:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 54 00000100:00100000:2.0:1789762721.921316:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067255168 00000100:00100000:2.0:1789762721.921326:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067255168 00002000:00010000:2.0:1789762721.921363:0:13301:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1789762721.921369:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff862680 with x1876702067255168 ext(0->18446744073709551615) 00002000:00010000:2.0:1789762721.921380:0:13301:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff862680. 00000100:00100000:2.0:1789762721.921402:0:13301:0:(nrs_tbf.c:1585:nrs_tbf_cli_findadd()) unknown.0 NRS: TBF class insert: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 3 deadline 0 time 195168476981 state 0x0 ref 1 00000100:00100000:2.0:1789762721.921414:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 3 deadline 195368476981 time 195168476981 state 0x1 ref 1 00000100:00100000:2.0:1789762721.921464:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067255296 00000100:00100000:2.0:1789762721.921470:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067255296 00002000:00010000:2.0:1789762721.921506:0:13301:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161088/4 (0->18446744073709551615) 00002000:00100000:2.0:1789762721.921509:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee70f100 with x1876702067255296 ext(0->18446744073709551615) 00000100:00100000:3.0:1789762721.921515:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 2 deadline 195368476981 time 195168590283 state 0x0 ref 1 00002000:00010000:2.0:1789762721.921530:0:13301:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ee70f100. 00000100:00100000:3.0:1789762721.921542:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff862680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2398:x1876702067255168:12345-192.168.203.37@tcp:10:dd.500 00000100:00100000:2.0:1789762721.921545:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 2 deadline 195368590283 time 195168590283 state 0x1 ref 2 00000100:00100000:2.0:1789762721.921578:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 1 deadline 195368590283 time 195168657814 state 0x0 ref 2 00000100:00100000:2.0:1789762721.921614:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2397:x1876702067255296:12345-192.168.203.37@tcp:10:dd.500 00002000:00010000:3.0:1789762721.922059:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1789762721.922068:0:8058:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1789762721 -> 1789762721 00002000:00010000:2.0:1789762721.922073:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1789762721.922075:0:8058:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1789762721 -> 1789762721 00002000:00010000:2.0:1789762721.922084:0:13301:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb mtime from disk: 1789762721 -> 1789762721 00002000:00010000:3.0:1789762721.922088:0:8058:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1789762721 -> 1789762721 00002000:00010000:2.0:1789762721.922090:0:13301:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb atime from disk: 1789762721 -> 1789762721 00002000:00010000:3.0:1789762721.922098:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 1 -> 1 00002000:00010000:2.0:1789762721.922099:0:13301:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb ctime from disk: 1789762721 -> 1789762721 00002000:00010000:2.0:1789762721.922107:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:3.0:1789762721.922175:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff862680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:2398:x1876702067255168:12345-192.168.203.37@tcp:10:dd.500 Request processed in 639us (949us total) trans 4294967433 rc 0/0 00000100:00100000:2.0:1789762721.922204:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:2397:x1876702067255296:12345-192.168.203.37@tcp:10:dd.500 Request processed in 592us (790us total) trans 4294967327 rc 0/0 00000100:00100000:3.0:1789762721.922216:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 0 00002000:00010000:3.0:1789762721.922227:0:8058:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419264/4 (0->18446744073709551615) 00000100:00100000:2.0:1789762721.922233:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1 00002000:00100000:3.0:1789762721.922240:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff862680 with x1876702067255168 ext(0->18446744073709551615) 00000100:00100000:2.0:1789762721.922245:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 1 deadline 195368590283 time 195168657814 state 0x2 ref 0 00002000:00010000:2.0:1789762721.922253:0:13301:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161088/4 (0->18446744073709551615) 00002000:00100000:2.0:1789762721.922264:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee70f100 with x1876702067255296 ext(0->18446744073709551615) 00002000:00010000:3.0:1789762721.922272:0:8058:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff862680. 00002000:00010000:2.0:1789762721.922273:0:13301:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ee70f100. 00000100:00100000:2.0:1789762721.957129:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762721.957226:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067255680 00000100:00100000:3.0:1789762721.957234:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067255680 00002000:00100000:3.0:1789762721.957264:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (2097152->3145727) req@ffff9c37e3b8ea00 x1876702067255680/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:502/0 lens 488/0 e 0 to 0 dl 1789762732 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762721.957277:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b8ea00 with x1876702067255680 ext(2097152->3145727) 00002000:00010000:3.0:1789762721.957282:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e3b8ea00 00000100:00100000:3.0:1789762721.957292:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 195383838497 time 195183838497 state 0x1 ref 2 00000100:00100000:2.0:1789762721.977631:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067255936 00000100:00100000:2.0:1789762721.977639:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067255936 00002000:00100000:2.0:1789762721.977670:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x4:0x0] (0->1048575) req@ffff9c37e3b94700 x1876702067255936/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:502/0 lens 488/0 e 0 to 0 dl 1789762732 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762721.977680:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b94700 with x1876702067255936 ext(0->1048575) 00002000:00010000:2.0:1789762721.977684:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3b94700 00000100:00100000:2.0:1789762721.977701:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 195383838497 time 195183838497 state 0x1 ref 4 00000100:00100000:2.0:1789762721.987383:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762722.137002:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 195584068278 time 195384068278 state 0x1 ref 5 00000100:00100000:2.0:1789762722.137038:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d174bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:2396:x1876702067255552:12345-192.168.203.37@tcp:4:dd.500 00010000:00010000:3.0:1789762722.322216:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1742e3 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322244:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1742e3 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322271:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1742e3 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1789762722.322283:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 3 committed batchid 2 00010000:00010000:3.0:1789762722.322288:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cfe00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762722.322301:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1742e3 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322321:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1742e3 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322376:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x2b0e463d (ffff9c37d3beed00) 00010000:00010000:3.0:1789762722.322384:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762722.322395:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1742e3 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322469:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1742e3 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322537:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934200/0xdf5a7ac59f1742f8 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1789762722.322546:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 2 00010000:00010000:3.0:1789762722.322579:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934200/0xdf5a7ac59f1742f8 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322621:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934200/0xdf5a7ac59f1742f8 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322643:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762722.322655:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934200/0xdf5a7ac59f1742f8 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322681:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934200/0xdf5a7ac59f1742f8 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322725:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x8:0x0].0x0 (ffff9c37c72ba900) 00010000:00010000:3.0:1789762722.322730:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762722.322738:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934200/0xdf5a7ac59f1742f8 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322770:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934200/0xdf5a7ac59f1742f8 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322791:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17434c lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322806:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17434c lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322855:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17434c lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322894:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8ce800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762722.322902:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17434c lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322911:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17434c lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.322976:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7:0x0].0x0 (ffff9c37db743300) 00010000:00010000:3.0:1789762722.322992:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789762722.323003:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17434c lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323043:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f17434c lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323100:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935600/0xdf5a7ac59f17437d lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323120:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935600/0xdf5a7ac59f17437d lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323167:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935600/0xdf5a7ac59f17437d lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323189:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762722.323204:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935600/0xdf5a7ac59f17437d lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323257:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935600/0xdf5a7ac59f17437d lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323302:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x8:0x0].0x0 (ffff9c37c72ba900) 00010000:00010000:3.0:1789762722.323308:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789762722.323314:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935600/0xdf5a7ac59f17437d lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323324:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935600/0xdf5a7ac59f17437d lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323381:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f1743a7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323400:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f1743a7 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323453:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f1743a7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323469:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762722.323485:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f1743a7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323520:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f1743a7 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323555:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7:0x0].0x0 (ffff9c37db743300) 00010000:00010000:3.0:1789762722.323566:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762722.323581:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f1743a7 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762722.323647:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f1743a7 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762722.337140:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 195784207052 time 195584207052 state 0x1 ref 4 00000100:00100000:2.0:1789762722.337182:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+16:2396:x1876702067255680:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762722.537303:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 195984349845 time 195784349845 state 0x1 ref 3 00000100:00100000:3.0:1789762722.537343:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d174a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:2396:x1876702067255808:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762722.541677:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d174a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:2396:x1876702067255808:12345-192.168.203.37@tcp:4:dd.500 Request processed in 4350us (574209us total) trans 4294967437 rc 0/0 00000100:00100000:3.0:1789762722.541690:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 5 00002000:00100000:3.0:1789762722.541703:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (3145728->4194303) req@ffff9c37d174a300 x1876702067255808/t4294967437(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:502/0 lens 488/448 e 0 to 0 dl 1789762732 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762722.541715:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d174a300 with x1876702067255808 ext(3145728->4194303) 00002000:00010000:3.0:1789762722.541723:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d174a300 00000100:00100000:2.0:1789762722.748266:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762722.774210:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762722.781119:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067256704 00000100:00100000:3.0:1789762722.781126:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067256704 00002000:00100000:3.0:1789762722.781154:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x4:0x0] (6291456->7340031) req@ffff9c37f7a37100 x1876702067256704/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:503/0 lens 488/0 e 0 to 0 dl 1789762733 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762722.781167:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f7a37100 with x1876702067256704 ext(6291456->7340031) 00002000:00010000:3.0:1789762722.781172:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37f7a37100 00000100:00100000:3.0:1789762722.781182:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 196184483870 time 195984483870 state 0x1 ref 6 00000100:00100000:2.0:1789762722.937544:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 196384615502 time 196184615502 state 0x1 ref 7 00000100:00100000:2.0:1789762722.937575:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:2396:x1876702067256064:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762722.940915:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:2396:x1876702067256064:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3353us (953531us total) trans 4294967329 rc 0/0 00000100:00100000:2.0:1789762722.940963:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 7 00002000:00100000:2.0:1789762722.940975:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x4:0x0] (1048576->2097151) req@ffff9c37e3b8ce00 x1876702067256064/t4294967329(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:502/0 lens 488/448 e 0 to 0 dl 1789762732 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762722.940986:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b8ce00 with x1876702067256064 ext(1048576->2097151) 00002000:00010000:2.0:1789762722.940992:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3b8ce00 00000100:00100000:2.0:1789762723.137608:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 196584679550 time 196384679550 state 0x1 ref 8 00000100:00100000:2.0:1789762723.137645:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:2396:x1876702067256192:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:3.0:1789762723.137670:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762723.140343:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:2396:x1876702067256192:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2720us (392079us total) trans 4294967330 rc 0/0 00000100:00100000:2.0:1789762723.140351:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 8 00002000:00100000:2.0:1789762723.140360:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x4:0x0] (2097152->3145727) req@ffff9c37e3b8d500 x1876702067256192/t4294967330(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:503/0 lens 488/448 e 0 to 0 dl 1789762733 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762723.140367:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b8d500 with x1876702067256192 ext(2097152->3145727) 00002000:00010000:2.0:1789762723.140372:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3b8d500 00000100:00100000:2.0:1789762723.152642:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067257344 00000100:00100000:2.0:1789762723.152649:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067257344 00002000:00100000:2.0:1789762723.152689:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0x4:0x0] (7340032->8388607) req@ffff9c37ee70fb80 x1876702067257344/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/0 e 0 to 0 dl 1789762734 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762723.152700:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee70fb80 with x1876702067257344 ext(7340032->8388607) 00002000:00010000:2.0:1789762723.152705:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ee70fb80 00000100:00100000:2.0:1789762723.152716:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 196584679550 time 196384679550 state 0x1 ref 9 00000100:00100000:2.0:1789762723.152956:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067257600 00000100:00100000:2.0:1789762723.152962:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067257600 00000100:00100000:2.0:1789762723.153027:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 28 00000100:00100000:2.0:1789762723.153039:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbd850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:12094:x1876702067257600:12345-192.168.203.37@tcp:35:dd.500 00000100:00080000:3.0:1789762723.153100:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000001:00080000:2.0:1789762723.153483:0:6422:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967345 00000100:00100000:2.0:1789762723.153597:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbd850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:12094:x1876702067257600:12345-192.168.203.37@tcp:35:dd.500 Request processed in 561us (716us total) trans 4294967346 rc 0/0 00000100:00100000:2.0:1789762723.153604:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 28 00000020:00080000:2.0:1789762723.153609:0:6422:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789762723.188336:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067257856 00000100:00100000:2.0:1789762723.188360:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067257856 00000100:00100000:2.0:1789762723.188411:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 127 00000100:00100000:2.0:1789762723.188436:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee70d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:12105:x1876702067257856:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762723.188458:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762723.188477:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f1743bc lrc: 2/0,0 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c457 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.188793:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f1743c3 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.188806:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f1743c3 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.188816:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f1743c3 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.188827:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f1743ca lrc: 3/1,0 mode: --/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.188832:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f1743ca lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.188836:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f1743ca lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189002:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1743d1 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189009:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1743d1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189014:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1743d1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762723.189027:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x8:0x0]: returning size 0 00010000:00010000:2.0:1789762723.189105:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f1743c3 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189111:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f1743c3 lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189116:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f1743c3 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189120:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cea00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762723.189123:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f1743c3 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189129:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f1743c3 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189140:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:2.0:1789762723.189143:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762723.189148:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f1743c3 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189155:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f1743c3 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189185:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f1743ca lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189192:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f1743ca lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189211:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f1743ca lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189216:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8ce800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762723.189219:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f1743ca lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189223:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f1743ca lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189229:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x2b0e463d (ffff9c37d3bee000) 00010000:00010000:2.0:1789762723.189231:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762723.189233:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f1743ca lrc: 2/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189237:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce800/0xdf5a7ac59f1743ca lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189272:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8ce200/0xdf5a7ac59f1743bc lrc: 0/0,0 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c457 expref: 13 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189282:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1743d1 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c457 expref: 12 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.189288:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x8:0x0].0x0 (ffff9c37c72ba900) 00010000:00010000:2.0:1789762723.189289:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762723.189291:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8cfe00, rc 0) 00000100:00100000:2.0:1789762723.189339:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee70d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:12105:x1876702067257856:12345-192.168.203.37@tcp:101:dd.0 Request processed in 919us (1082us total) trans 4294967347 rc 0/0 00000100:00100000:2.0:1789762723.189351:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 127 00000100:00100000:3.0:1789762723.196650:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067257984 00000100:00100000:3.0:1789762723.196656:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067257984 00000100:00100000:3.0:1789762723.196695:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 81 00000100:00100000:3.0:1789762723.196703:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:12105:x1876702067257984:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762723.196712:0:8052:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762723.196721:0:8052:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f1721668/0xdf5a7ac59f1743df lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c45e expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.196740:0:8052:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f976f348/0xdf5a7ac59f17436f lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c434 expref: 23 pid: 8054 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.196748:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762723.196751:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762723.196755:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f1721668/0xdf5a7ac59f1743df lrc: 4/0,0 mode: --/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c45e expref: 23 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.196849:0:8052:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f976f348/0xdf5a7ac59f17436f lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c434 expref: 23 pid: 8054 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.196862:0:8052:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f976f348/0xdf5a7ac59f17436f lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c434 expref: 23 pid: 8054 timeout: 296 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762723.196873:0:8052:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f63af480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8052:1876702074610688:192.168.203.37@tcp:104: 00000100:00100000:2.0:1789762723.197032:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067258112 00000100:00100000:2.0:1789762723.197038:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067258112 00000100:00100000:3.0:1789762723.197063:0:8052:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d24be6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762723.197081:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067258240 00000100:00100000:2.0:1789762723.197085:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067258240 00000100:00100000:2.0:1789762723.197124:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 83 00000100:00100000:2.0:1789762723.197139:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:12107:x1876702067258240:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762723.197155:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762723.197168:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ccca588f/0xdf5a7ac59f1743ed lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c46c expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.197202:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762723.197210:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762723.197220:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ccca588f/0xdf5a7ac59f1743ed lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c46c expref: 27 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.197233:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ccca588f/0xdf5a7ac59f1743ed lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c46c expref: 27 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.197246:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8ce200, rc 0) 00000100:00100000:2.0:1789762723.197285:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:12107:x1876702067258240:12345-192.168.203.37@tcp:101:dd.0 Request processed in 149us (235us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762723.197306:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 83 00000100:00100000:3.0:1789762723.197711:0:8052:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f63af480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8052:1876702074610688:192.168.203.37@tcp:104: 00010000:00010000:3.0:1789762723.197728:0:8052:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f1721668/0xdf5a7ac59f1743df lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 5 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c45e expref: 25 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.197750:0:8052:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff935600, rc 0) 00000100:00100000:3.0:1789762723.197778:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:12105:x1876702067257984:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1077us (1170us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762723.197784:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 81 00000100:00100000:3.0:1789762723.218037:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067258496 00000100:00100000:3.0:1789762723.218047:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067258496 00000100:00100000:3.0:1789762723.218117:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 84 00000100:00100000:3.0:1789762723.218192:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ebaa3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:12109:x1876702067258496:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762723.218208:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762723.218224:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c473 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.218251:0:8054:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174376 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c43b expref: 24 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.218258:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762723.218262:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762723.218266:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 24 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.218299:0:8054:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174376 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c43b expref: 24 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.218315:0:8054:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174376 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c43b expref: 24 pid: 8053 timeout: 296 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762723.218383:0:8054:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f63af480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8054:1876702074611200:192.168.203.37@tcp:104: 00000100:00100000:3.0:1789762723.218480:0:8054:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d24be6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762723.218878:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067258752 00000100:00100000:3.0:1789762723.218883:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067258752 00000100:00100000:3.0:1789762723.218937:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 86 00000100:00100000:3.0:1789762723.218947:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:12117:x1876702067258752:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762723.218955:0:8052:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762723.218962:0:8052:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f174402 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c481 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.218975:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762723.218979:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762723.218982:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f174402 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c481 expref: 27 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.219040:0:8052:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f174402 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c481 expref: 27 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.219052:0:8052:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff934e00, rc 0) 00000100:00100000:3.0:1789762723.219084:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:12117:x1876702067258752:12345-192.168.203.37@tcp:101:dd.0 Request processed in 140us (254us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762723.219089:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 86 00000100:00100000:3.0:1789762723.219414:0:8054:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f63af480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8054:1876702074611200:192.168.203.37@tcp:104: 00010000:00010000:3.0:1789762723.219445:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 26 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.219460:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff934200, rc 0) 00000100:00100000:3.0:1789762723.219484:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:12109:x1876702067258496:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1299us (1516us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762723.219489:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 84 00000100:00100000:3.0:1789762723.219773:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067258880 00000100:00100000:3.0:1789762723.219777:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067258880 00010000:00010000:3.0:1789762723.219791:0:7218:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174376 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c43b expref: 26 pid: 8053 timeout: 296 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762723.219814:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 56 00000100:00100000:3.0:1789762723.219823:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb931880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2396:x1876702067258880:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762723.219836:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762723.219858:0:7218:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174376 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c43b expref: 26 pid: 8053 timeout: 296 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.219871:0:7218:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174376 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c43b expref: 26 pid: 8053 timeout: 296 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.219887:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174376 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c43b expref: 26 pid: 8053 timeout: 296 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.219896:0:7218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762723.219899:0:7218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762723.219905:0:7218:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.219954:0:7218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.219962:0:7218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762723.219964:0:7218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762723.219969:0:7218:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f1743fb lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [5242880->18446744073709551615] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c47a expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.219976:0:7218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f1743fb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [5242880->18446744073709551615] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c47a expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.219981:0:7218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762723.219983:0:7218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762723.219987:0:7218:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f174402 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c481 expref: 25 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.219994:0:7218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f174402 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c481 expref: 25 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.220010:0:7218:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f174402 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c481 expref: 25 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762723.220017:0:7218:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f63af480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7218:1876702074611328:192.168.203.37@tcp:105: 00010000:00010000:3.0:1789762723.220054:0:7218:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f1743fb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [5242880->18446744073709551615] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c47a expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762723.220069:0:7218:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f63ad180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7218:1876702074611456:192.168.203.37@tcp:105: 00010000:00010000:3.0:1789762723.220096:0:7218:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762723.220108:0:7218:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bc6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7218:1876702074611584:192.168.203.37@tcp:105: 00000100:00100000:3.0:1789762723.220237:0:7218:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d24be6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762723.221182:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067259008 00000100:00100000:2.0:1789762723.221189:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067259008 00000100:00100000:3.0:1789762723.221206:0:7218:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f63af480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7218:1876702074611328:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762723.221243:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 87 00000100:00100000:2.0:1789762723.221252:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:12111:x1876702067259008:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762723.221269:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762723.221278:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff23cf49/0xdf5a7ac59f174409 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c488 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.221299:0:8053:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 6 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 27 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.221306:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762723.221309:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789762723.221312:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff23cf49/0xdf5a7ac59f174409 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c488 expref: 27 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.221341:0:8053:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 6 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 27 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.221351:0:8053:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 6 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 27 pid: 8054 timeout: 296 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762723.221361:0:8053:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b8d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074611712:192.168.203.37@tcp:104: 00000100:00100000:2.0:1789762723.221431:0:8053:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7fbe9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762723.221819:0:7218:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f63ad180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7218:1876702074611456:192.168.203.37@tcp:105: 00000100:00100000:3.0:1789762723.221832:0:7218:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bc6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7218:1876702074611584:192.168.203.37@tcp:105: 00010000:00010000:3.0:1789762723.221851:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762723.221886:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb931880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2396:x1876702067258880:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 2063us (2141us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762723.221895:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 56 00000100:00100000:2.0:1789762723.222226:0:8053:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b8d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074611712:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762723.222245:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ff23cf49/0xdf5a7ac59f174409 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c488 expref: 26 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.222254:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8ce800, rc 0) 00000100:00100000:2.0:1789762723.222292:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:12111:x1876702067259008:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1041us (1162us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762723.222299:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 87 00000100:00100000:3.0:1789762723.247753:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067259264 00000100:00100000:3.0:1789762723.247762:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067259264 00000100:00100000:3.0:1789762723.247812:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 89 00000100:00100000:3.0:1789762723.247821:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:12112:x1876702067259264:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762723.247843:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762723.247859:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f174417 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c496 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.247885:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x5:0x0].0x0 since=0s v=(2 0) 00010000:00010000:3.0:1789762723.247902:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(3 0) 00010000:00010000:3.0:1789762723.247909:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f174417 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c496 expref: 28 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.247955:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f174417 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c496 expref: 28 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762723.247970:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff935400, rc 0) 00000100:00100000:3.0:1789762723.248019:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:12112:x1876702067259264:12345-192.168.203.37@tcp:101:dd.0 Request processed in 198us (321us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762723.248027:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 89 00000100:00100000:2.0:1789762723.309604:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067259392 00000100:00100000:2.0:1789762723.309628:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067259392 00002000:00100000:2.0:1789762723.309696:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0x5:0x0] (7340032->8388607) req@ffff9c37c7831f80 x1876702067259392/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/0 e 0 to 0 dl 1789762734 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762723.309707:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c7831f80 with x1876702067259392 ext(7340032->8388607) 00002000:00010000:2.0:1789762723.309713:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c7831f80 00000100:00100000:2.0:1789762723.309739:0:13301:0:(nrs_tbf.c:1585:nrs_tbf_cli_findadd()) unknown.0 NRS: TBF class insert: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 3 deadline 0 time 196556814092 state 0x0 ref 1 00000100:00100000:2.0:1789762723.309763:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 3 deadline 196556914092 time 196556814092 state 0x1 ref 1 00000100:00100000:2.0:1789762723.309871:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196556914092 time 196556949148 state 0x0 ref 1 00000100:00100000:2.0:1789762723.309885:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7831f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2398:x1876702067259392:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762723.315484:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067259520 00000100:00100000:2.0:1789762723.315489:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067259520 00002000:00100000:2.0:1789762723.315510:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x5:0x0] (0->1048575) req@ffff9c37c7830000 x1876702067259520/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/0 e 0 to 0 dl 1789762734 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762723.315518:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c7830000 with x1876702067259520 ext(0->1048575) 00010000:00010000:2.0:1789762723.315526:0:13300:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 30 pid: 8054 timeout: 296 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.315541:0:13300:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 30 pid: 8054 timeout: 296 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762723.315559:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9c37c7830000 00000100:00100000:2.0:1789762723.315570:0:13300:0:(nrs_tbf.c:1585:nrs_tbf_cli_findadd()) unknown.0 NRS: TBF class insert: class@ffff9c37ee4a6e00 key gid={0} rule@0xffff9c37c7d30800 gen 0 rate 10000 depth 3 token 3 deadline 0 time 196562649626 state 0x0 ref 1 00000100:00100000:2.0:1789762723.315576:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ee4a6e00 key gid={0} rule@0xffff9c37c7d30800 gen 0 rate 10000 depth 3 token 3 deadline 196562749626 time 196562649626 state 0x1 ref 1 00000100:00100000:2.0:1789762723.315596:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ee4a6e00 key gid={0} rule@0xffff9c37c7d30800 gen 0 rate 10000 depth 3 token 2 deadline 196562749626 time 196562676302 state 0x0 ref 1 00000100:00100000:2.0:1789762723.315610:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7830000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:2397:x1876702067259520:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762723.319219:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7830000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2397:x1876702067259520:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3611us (3804us total) trans 4294967439 rc 0/0 00000100:00100000:2.0:1789762723.319231:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 0 00000100:00100000:2.0:1789762723.319236:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ee4a6e00 key gid={0} rule@0xffff9c37c7d30800 gen 0 rate 10000 depth 3 token 2 deadline 196562749626 time 196562676302 state 0x2 ref 0 00002000:00100000:2.0:1789762723.319244:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x5:0x0] (0->1048575) req@ffff9c37c7830000 x1876702067259520/t4294967439(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/448 e 0 to 0 dl 1789762734 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762723.319251:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c7830000 with x1876702067259520 ext(0->1048575) 00010000:00010000:2.0:1789762723.319257:0:13300:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 32 pid: 8054 timeout: 296 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.319276:0:13300:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 32 pid: 8054 timeout: 296 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762723.319283:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9c37c7830000 00000100:00100000:3.0:1789762723.320545:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762723.320591:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067259648 00000100:00100000:2.0:1789762723.320596:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067259648 00002000:00100000:2.0:1789762723.320622:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0x5:0x0] (6291456->7340031) req@ffff9c37e3bc6d80 x1876702067259648/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/0 e 0 to 0 dl 1789762734 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762723.320630:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bc6d80 with x1876702067259648 ext(6291456->7340031) 00002000:00010000:2.0:1789762723.320633:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e3bc6d80 00000100:00100000:2.0:1789762723.320638:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196556914092 time 196556949148 state 0x0 ref 1 00000100:00100000:2.0:1789762723.320668:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196557049148 time 196556949148 state 0x1 ref 1 00000100:00100000:2.0:1789762723.320692:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196557049148 time 196567771918 state 0x0 ref 1 00000100:00100000:2.0:1789762723.320702:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2395:x1876702067259648:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762723.322363:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762723.322435:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067259776 00000100:00100000:2.0:1789762723.322442:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067259776 00010000:00010000:2.0:1789762723.322474:0:6405:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 34 pid: 8054 timeout: 296 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762723.322515:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 57 00000100:00100000:2.0:1789762723.322525:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2398:x1876702067259776:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762723.322539:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1789762723.322552:0:6405:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:2.0:1789762723.322555:0:6405:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb mtime from disk: 1789762721 -> 1789762723 00002000:00010000:2.0:1789762723.322558:0:6405:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb ctime from disk: 1789762721 -> 1789762723 00010000:00010000:2.0:1789762723.322564:0:6405:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 8 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 34 pid: 8054 timeout: 296 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.322576:0:6405:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 8 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 34 pid: 8054 timeout: 296 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.322607:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1743f4 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 8 type: EXT [0->4194303] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c473 expref: 34 pid: 8054 timeout: 296 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.322623:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762723.322628:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762723.322636:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ff23cf49/0xdf5a7ac59f174409 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c488 expref: 33 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.322646:0:6405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff23cf49/0xdf5a7ac59f174409 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c488 expref: 33 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.322653:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762723.322655:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762723.322659:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174410 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c48f expref: 33 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.322665:0:6405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174410 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c48f expref: 33 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.322671:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762723.322673:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762723.322677:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f174417 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c496 expref: 33 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.322683:0:6405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f174417 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c496 expref: 33 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762723.322723:0:6405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f174417 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c496 expref: 33 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762723.322737:0:6405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b8d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6405:1876702074611840:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762723.322850:0:6405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174410 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c48f expref: 33 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762723.322862:0:6405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b8ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6405:1876702074611968:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762723.322904:0:6405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ff23cf49/0xdf5a7ac59f174409 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c488 expref: 33 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762723.322913:0:6405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b8f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6405:1876702074612096:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762723.323028:0:6405:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7fbe9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762723.328045:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ca315180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074612224:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762723.328088:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762723.328402:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ca315180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074612224:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762723.328882:0:6405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b8d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6405:1876702074611840:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762723.328903:0:6405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b8ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6405:1876702074611968:192.168.203.37@tcp:105: 00000100:00100000:3.0:1789762723.328941:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067259904 00000100:00100000:3.0:1789762723.328950:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067259904 00000100:00100000:2.0:1789762723.328954:0:6405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b8f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6405:1876702074612096:192.168.203.37@tcp:105: 00002000:00100000:3.0:1789762723.328994:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x5:0x0] (4194304->5242879) req@ffff9c37fdfce680 x1876702067259904/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/0 e 0 to 0 dl 1789762734 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00010000:00010000:2.0:1789762723.329004:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00002000:00100000:3.0:1789762723.329009:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfce680 with x1876702067259904 ext(4194304->5242879) 00002000:00010000:3.0:1789762723.329017:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fdfce680 00000100:00100000:3.0:1789762723.329029:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196557049148 time 196567771918 state 0x0 ref 1 00000100:00100000:2.0:1789762723.329067:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2398:x1876702067259776:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 6538us (6702us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762723.329072:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196567871918 time 196567771918 state 0x1 ref 1 00000100:00100000:2.0:1789762723.329103:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 57 00000100:00100000:3.0:1789762723.329123:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196576301595 time 196576201595 state 0x1 ref 2 00000100:00100000:3.0:1789762723.329150:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2396:x1876702067259904:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762723.333057:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2396:x1876702067259904:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3914us (4190us total) trans 4294967442 rc 0/0 00000100:00100000:3.0:1789762723.333063:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 20 00000100:00100000:3.0:1789762723.333067:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 1 deadline 196576301595 time 196576224326 state 0x2 ref 0 00002000:00100000:3.0:1789762723.333074:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x5:0x0] (4194304->5242879) req@ffff9c37fdfce680 x1876702067259904/t4294967442(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/448 e 0 to 0 dl 1789762734 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762723.333080:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfce680 with x1876702067259904 ext(4194304->5242879) 00002000:00010000:3.0:1789762723.333083:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fdfce680 00000100:00100000:3.0:1789762723.340470:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2398:x1876702067256320:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2784us (585494us total) trans 4294967331 rc 0/0 00000100:00100000:3.0:1789762723.340478:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 9 00002000:00100000:3.0:1789762723.340488:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x4:0x0] (3145728->4194303) req@ffff9c37e3b94e00 x1876702067256320/t4294967331(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:503/0 lens 488/448 e 0 to 0 dl 1789762733 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762723.340495:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b94e00 with x1876702067256320 ext(3145728->4194303) 00002000:00010000:3.0:1789762723.340499:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3b94e00 00000100:00100000:2.0:1789762723.349834:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762723.349869:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762723.349901:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067260160 00000100:00100000:3.0:1789762723.349913:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067260160 00002000:00100000:3.0:1789762723.349996:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0x5:0x0] (1048576->2097151) req@ffff9c37e3b8f100 x1876702067260160/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/0 e 0 to 0 dl 1789762734 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762723.350023:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b8f100 with x1876702067260160 ext(1048576->2097151) 00002000:00010000:3.0:1789762723.350033:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3b8f100 00000100:00100000:3.0:1789762723.350048:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 1 deadline 196576301595 time 196576224326 state 0x0 ref 1 00000100:00100000:3.0:1789762723.350115:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 1 deadline 196576324326 time 196576224326 state 0x1 ref 2 00000100:00100000:3.0:1789762723.350906:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762723.351054:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067260416 00000100:00100000:3.0:1789762723.351062:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067260416 00002000:00100000:3.0:1789762723.351108:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x5:0x0] (3145728->4194303) req@ffff9c37e3bc4a80 x1876702067260416/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/0 e 0 to 0 dl 1789762734 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762723.351124:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bc4a80 with x1876702067260416 ext(3145728->4194303) 00002000:00010000:3.0:1789762723.351131:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e3bc4a80 00000100:00100000:3.0:1789762723.351189:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 1 deadline 196597335190 time 196597235190 state 0x1 ref 3 00000100:00100000:3.0:1789762723.351219:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196597335190 time 196598298336 state 0x0 ref 3 00000100:00100000:3.0:1789762723.351242:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2398:x1876702067260416:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762723.351248:0:8057:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 5 min 3 max 116 00000100:00100000:2.0:1789762723.351266:0:8057:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_005' 00000100:00100000:3.0:1789762723.354230:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762723.359475:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762723.360525:0:14170:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 6 min 3 max 116 00000100:00100000:3.0:1789762723.360577:0:14170:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_006' 00000100:00100000:2.0:1789762723.365866:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2397:x1876702067260288:12345-192.168.203.37@tcp:4:dd.0 Request processed in 15696us (15981us total) trans 4294967333 rc 0/0 00000100:00100000:2.0:1789762723.365880:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 22 00002000:00100000:2.0:1789762723.365893:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x5:0x0] (2097152->3145727) req@ffff9c37e3b8ca80 x1876702067260288/t4294967333(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/448 e 0 to 0 dl 1789762734 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762723.365901:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b8ca80 with x1876702067260288 ext(2097152->3145727) 00002000:00010000:2.0:1789762723.365905:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3b8ca80 00000100:00100000:3.0:1789762723.375781:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067261056 00000100:00100000:3.0:1789762723.375791:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067261056 00000100:00100000:2.0:1789762723.375796:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067260928 00000100:00100000:2.0:1789762723.375803:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067260928 00002000:00100000:3.0:1789762723.375840:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x5:0x0] (0->1048575) req@ffff9c37fdfc7480 x1876702067261056/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/0 e 0 to 0 dl 1789762734 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762723.375841:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0x5:0x0] (5242880->6291455) req@ffff9c37fdfc5180 x1876702067260928/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/0 e 0 to 0 dl 1789762734 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762723.375899:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfc5180 with x1876702067260928 ext(5242880->6291455) 00002000:00100000:3.0:1789762723.375902:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfc7480 with x1876702067261056 ext(0->1048575) 00002000:00010000:2.0:1789762723.375906:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fdfc5180 00002000:00010000:3.0:1789762723.375980:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fdfc7480 00000100:00100000:2.0:1789762723.375997:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196607681694 time 196607581694 state 0x1 ref 4 00000100:00100000:3.0:1789762723.376017:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196607681694 time 196607581694 state 0x1 ref 4 00000100:00100000:2.0:1789762723.376042:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 1 deadline 196623220136 time 196623120136 state 0x1 ref 4 00000100:00100000:2.0:1789762723.376070:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2398:x1876702067260928:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762723.376197:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 0 deadline 196623239749 time 196623275263 state 0x0 ref 4 00000100:00100000:3.0:1789762723.376226:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:2398:x1876702067261056:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762723.382345:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762723.382387:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067261440 00000100:00100000:2.0:1789762723.382393:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067261440 00000100:00100000:2.0:1789762723.382437:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 29 00000100:00100000:2.0:1789762723.382445:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f63af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:12108:x1876702067261440:12345-192.168.203.37@tcp:35:dd.0 00000001:00080000:2.0:1789762723.382609:0:13186:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967348 00000100:00100000:2.0:1789762723.382681:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f63af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:12108:x1876702067261440:12345-192.168.203.37@tcp:35:dd.0 Request processed in 237us (336us total) trans 4294967349 rc 0/0 00000100:00100000:2.0:1789762723.382687:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 29 00000020:00080000:2.0:1789762723.382691:0:13186:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789762723.384462:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762723.384505:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067261568 00000100:00100000:3.0:1789762723.384510:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067261568 00002000:00100000:3.0:1789762723.384533:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x5:0x0] (7340032->8388607) req@ffff9c37e3b8e680 x1876702067261568/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/0 e 0 to 0 dl 1789762734 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762723.384541:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b8e680 with x1876702067261568 ext(7340032->8388607) 00002000:00010000:3.0:1789762723.384544:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3b8e680 00000100:00100000:3.0:1789762723.384551:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 0 deadline 196623239749 time 196623275263 state 0x0 ref 1 00000100:00100000:3.0:1789762723.384577:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 0 deadline 196623375263 time 196623275263 state 0x1 ref 1 00000100:00100000:3.0:1789762723.384593:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196623375263 time 196631672435 state 0x0 ref 1 00000100:00100000:3.0:1789762723.384602:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2395:x1876702067261568:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762723.390468:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2395:x1876702067261568:12345-192.168.203.37@tcp:4:dd.0 Request processed in 5863us (6003us total) trans 4294967338 rc 0/0 00000100:00100000:3.0:1789762723.390479:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 31 00002000:00100000:3.0:1789762723.390490:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x5:0x0] (7340032->8388607) req@ffff9c37e3b8e680 x1876702067261568/t4294967338(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:504/0 lens 488/448 e 0 to 0 dl 1789762734 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762723.390501:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b8e680 with x1876702067261568 ext(7340032->8388607) 00002000:00010000:3.0:1789762723.390515:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3b8e680 00000100:00100000:2.0:1789762723.737888:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 197184952332 time 196984952332 state 0x1 ref 7 00000100:00100000:2.0:1789762723.737943:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:2398:x1876702067256576:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762724.338360:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 197785423903 time 197585423903 state 0x1 ref 4 00000100:00100000:2.0:1789762724.338440:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2395:x1876702067256960:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762724.343291:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2395:x1876702067256960:12345-192.168.203.37@tcp:4:dd.500 Request processed in 4898us (1399822us total) trans 4294967340 rc 0/0 00000100:00100000:2.0:1789762724.343306:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 14 00002000:00100000:2.0:1789762724.343322:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0x4:0x0] (4194304->5242879) req@ffff9c36c8edea00 x1876702067256960/t4294967340(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:503/0 lens 488/448 e 0 to 0 dl 1789762733 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762724.343338:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36c8edea00 with x1876702067256960 ext(4194304->5242879) 00002000:00010000:2.0:1789762724.343349:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36c8edea00 00000100:00100000:3.0:1789762724.352487:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074612352 00000100:00100000:3.0:1789762724.352502:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074612352 00000100:00100000:3.0:1789762724.352595:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:3.0:1789762724.352633:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074612352:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789762724.352683:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789762724.352773:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074612352:12345-0@lo:41:osp-pre-0-1.0 Request processed in 149us (400us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762724.352787:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00010000:00010000:3.0:1789762724.480196:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762724.538555:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 197985622390 time 197785622390 state 0x1 ref 3 00000100:00100000:3.0:1789762724.538596:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee70e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2395:x1876702067257088:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762724.543582:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee70e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:2395:x1876702067257088:12345-192.168.203.37@tcp:4:dd.500 Request processed in 4999us (1592548us total) trans 4294967341 rc 0/0 00000100:00100000:3.0:1789762724.543597:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 15 00002000:00100000:3.0:1789762724.543614:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x4:0x0] (5242880->6291455) req@ffff9c37ee70e680 x1876702067257088/t4294967341(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:503/0 lens 488/448 e 0 to 0 dl 1789762733 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762724.543629:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee70e680 with x1876702067257088 ext(5242880->6291455) 00002000:00010000:3.0:1789762724.543639:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ee70e680 00000100:00100000:3.0:1789762724.987996:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067261824 00000100:00100000:3.0:1789762724.988005:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067261824 00000100:00100000:3.0:1789762724.988059:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 15 00000100:00100000:3.0:1789762724.988071:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+47:6358:x1876702067261824:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762724.988088:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 6 locks, starting at 0 00002000:00010000:3.0:1789762724.988108:0:6406:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:3.0:1789762724.988113:0:6406:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1789762721 -> 1789762722 00002000:00010000:3.0:1789762724.988116:0:6406:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1789762721 -> 1789762722 00010000:00010000:3.0:1789762724.988141:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000628cf1d4/0xdf5a7ac59f1743ae lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c449 expref: 47 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762724.988186:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff23cf49/0xdf5a7ac59f174409 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 7 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c488 expref: 46 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762724.988196:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174410 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 6 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c48f expref: 45 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762724.988205:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f174417 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c496 expref: 44 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762724.988214:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f1743fb lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 4 type: EXT [5242880->18446744073709551615] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c47a expref: 43 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762724.988221:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f174402 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c481 expref: 42 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762724.988231:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762724.988276:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:6358:x1876702067261824:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 209us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762724.988281:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 15 00000100:00100000:2.0:1789762724.991136:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067261952 00000100:00100000:2.0:1789762724.991142:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067261952 00000100:00100000:2.0:1789762724.991177:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 16 00000100:00100000:2.0:1789762724.991187:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+45:6823:x1876702067261952:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762724.991197:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 4 locks, starting at 0 00002000:00010000:2.0:1789762724.991210:0:8070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:2.0:1789762724.991213:0:8070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb mtime from disk: 1789762721 -> 1789762722 00002000:00010000:2.0:1789762724.991222:0:8070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0x4:0x0] updating lvb ctime from disk: 1789762721 -> 1789762722 00010000:00010000:2.0:1789762724.991235:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ded696a4/0xdf5a7ac59f1743b5 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c450 expref: 45 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762724.991256:0:8070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0x5:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:2.0:1789762724.991257:0:8070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0x5:0x0] updating lvb mtime from disk: 1789762721 -> 1789762723 00002000:00010000:2.0:1789762724.991259:0:8070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0x5:0x0] updating lvb ctime from disk: 1789762721 -> 1789762723 00010000:00010000:2.0:1789762724.991265:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ccca588f/0xdf5a7ac59f1743ed lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 5 type: EXT [2097152->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c46c expref: 44 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762724.991274:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f1721668/0xdf5a7ac59f1743df lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c45e expref: 43 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762724.991282:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fd56c93e/0xdf5a7ac59f1743e6 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c465 expref: 42 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762724.991290:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762724.991317:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:6823:x1876702067261952:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 134us (233us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762724.991322:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 16 00000100:00100000:2.0:1789762725.024843:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.024894:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067262208 00000100:00100000:3.0:1789762725.024904:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067262208 00000100:00100000:3.0:1789762725.024991:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 128 00000100:00100000:3.0:1789762725.025003:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b8dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:12141:x1876702067262208:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762725.025026:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762725.025056:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f17441e lrc: 2/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c49d expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025117:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174425 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025127:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174425 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025137:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174425 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025147:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17442c lrc: 3/1,0 mode: --/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025152:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17442c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025156:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17442c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025401:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934600/0xdf5a7ac59f174433 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025422:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934600/0xdf5a7ac59f174433 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025427:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934600/0xdf5a7ac59f174433 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762725.025435:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x7:0x0]: returning size 16777216 00010000:00010000:3.0:1789762725.025518:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174425 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025531:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174425 lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025536:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174425 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025540:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762725.025543:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174425 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025549:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174425 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025559:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:3.0:1789762725.025561:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762725.025564:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174425 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025568:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f174425 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025578:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17442c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025583:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17442c lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025587:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17442c lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025590:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762725.025595:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17442c lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025599:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17442c lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025604:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x73550ecb (ffff9c37c72bae00) 00010000:00010000:3.0:1789762725.025606:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762725.025607:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17442c lrc: 2/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025611:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f17442c lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025639:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f17441e lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c49d expref: 17 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025649:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934600/0xdf5a7ac59f174433 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c49d expref: 16 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.025653:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7:0x0].0x0 (ffff9c37db743300) 00010000:00010000:3.0:1789762725.025655:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762725.025656:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff934600, rc 0) 00000100:00100000:3.0:1789762725.025704:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b8dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+16:12141:x1876702067262208:12345-192.168.203.37@tcp:101:dd.0 Request processed in 706us (870us total) trans 4294967350 rc 0/0 00000100:00100000:3.0:1789762725.025709:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 128 00000100:00100000:2.0:1789762725.027255:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067262336 00000100:00100000:2.0:1789762725.027261:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067262336 00000100:00100000:2.0:1789762725.027304:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 90 00000100:00100000:2.0:1789762725.027314:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:12141:x1876702067262336:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762725.027322:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762725.027340:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f174441 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c4a4 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762725.027359:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4:0x0] initial LVB size: 8388608, mtime: 0x6aad9ca2, atime: 0x6aad9ca1, ctime: 0x6aad9ca2, blocks: 0x1 00010000:00010000:2.0:1789762725.027371:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=198s v=(0 0) 00010000:00010000:2.0:1789762725.027375:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762725.027382:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f174441 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4a4 expref: 42 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.027392:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f174441 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4a4 expref: 42 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.027399:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8ce200, rc 0) 00000100:00100000:2.0:1789762725.027440:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:12141:x1876702067262336:12345-192.168.203.37@tcp:101:dd.0 Request processed in 128us (264us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762725.027445:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 90 00000100:00100000:2.0:1789762725.031795:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067262464 00000100:00100000:2.0:1789762725.031801:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067262464 00002000:00100000:2.0:1789762725.031826:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0x4:0x0] (0->1048575) req@ffff9c37d98a6a00 x1876702067262464/t0(0) o3->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:506/0 lens 488/0 e 0 to 0 dl 1789762736 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762725.031835:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d98a6a00 with x1876702067262464 ext(0->1048575) 00002000:00010000:2.0:1789762725.031839:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d98a6a00 00000100:00100000:2.0:1789762725.031848:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196631772435 time 196632561578 state 0x0 ref 1 00000100:00100000:2.0:1789762725.031856:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196632661578 time 196632561578 state 0x1 ref 1 00000100:00100000:2.0:1789762725.031876:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196632661578 time 198278955689 state 0x0 ref 1 00000100:00100000:2.0:1789762725.031892:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2398:x1876702067262464:12345-192.168.203.37@tcp:3:dd.0 00000100:00100000:2.0:1789762725.035610:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2398:x1876702067262464:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3717us (3862us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762725.035618:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 33 00000100:00100000:2.0:1789762725.035623:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196632661578 time 198278955689 state 0x2 ref 0 00002000:00100000:2.0:1789762725.035632:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0x4:0x0] (0->1048575) req@ffff9c37d98a6a00 x1876702067262464/t0(0) o3->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:506/0 lens 488/440 e 0 to 0 dl 1789762736 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762725.035639:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d98a6a00 with x1876702067262464 ext(0->1048575) 00002000:00010000:2.0:1789762725.035642:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d98a6a00 00000100:00100000:3.0:1789762725.076059:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.086672:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067262848 00000100:00100000:3.0:1789762725.086680:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067262848 00000100:00100000:3.0:1789762725.086726:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 129 00000100:00100000:3.0:1789762725.086746:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:12157:x1876702067262848:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:3.0:1789762725.086767:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762725.086784:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935200/0xdf5a7ac59f174448 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c4ab expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.086844:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935800/0xdf5a7ac59f17444f lrc: 3/1,0 mode: --/PR res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.086860:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935800/0xdf5a7ac59f17444f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.086873:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935800/0xdf5a7ac59f17444f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762725.086907:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x6:0x0]: returning size 16384 00000004:00200000:3.0:1789762725.086911:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762725.086955:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935800/0xdf5a7ac59f17444f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.086977:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935200/0xdf5a7ac59f174448 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4ab expref: 19 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.086995:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935800/0xdf5a7ac59f17444f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4ab expref: 18 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.087001:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:3.0:1789762725.087004:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789762725.087006:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff935800, rc 0) 00000100:00100000:3.0:1789762725.087052:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:12157:x1876702067262848:12345-192.168.203.37@tcp:101:rm.0 Request processed in 307us (431us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762725.087071:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 129 00000100:00100000:3.0:1789762725.088060:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762725.089831:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.089867:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067263232 00000100:00100000:3.0:1789762725.089872:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067263232 00000100:00100000:3.0:1789762725.089904:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 31 00000100:00100000:3.0:1789762725.089940:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b8f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:12157:x1876702067263232:12345-192.168.203.37@tcp:37:rm.0 00000100:00100000:3.0:1789762725.091769:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b8f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:12157:x1876702067263232:12345-192.168.203.37@tcp:37:rm.0 Request processed in 1854us (1935us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762725.091778:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 31 00000100:00100000:2.0:1789762725.094143:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00080000:0.0:1789762725.095541:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762725.096595:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762725.096638:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067264128 00000100:00100000:2.0:1789762725.096643:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067264128 00000100:00100000:2.0:1789762725.096672:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 133 00000100:00100000:2.0:1789762725.096679:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:12157:x1876702067264128:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:2.0:1789762725.096702:0:11692:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789762725.096714:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934600/0xdf5a7ac59f174433 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c49d expref: 22 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.096728:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031c00/0xdf5a7ac59f1742c0 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3f5 expref: 21 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.096735:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7:0x0].0x0 (ffff9c37db743300) 00010000:00010000:2.0:1789762725.096743:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935800/0xdf5a7ac59f17444f lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4ab expref: 20 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.096756:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:2.0:1789762725.096758:0:11692:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762725.096768:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174479 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.096774:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174479 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.096779:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174479 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.096787:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174480 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x6:0x0].0x73550ecb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.096792:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174480 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.096796:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174480 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.097005:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f174487 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.097012:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f174487 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.097017:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f174487 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789762725.097825:0:11692:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].4, 64 off8448 00000004:00080000:2.0:1789762725.097833:0:11692:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 198344914730ns, next at 199344913023ns, rc = 0 00000040:00080000:2.0:1789762725.097978:0:11692:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].4, 64 off8448 00000004:00080000:2.0:1789762725.097984:0:11692:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 198345065556ns, next at 199345064400ns, rc = 0 00010000:00010000:2.0:1789762725.098022:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f17448e lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098029:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f17448e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098035:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f17448e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098040:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f17448e lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098045:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f17448e lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098049:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f17448e lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098064:0:11692:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762725.098078:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f174487 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098085:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f174487 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098092:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f174487 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098098:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cee00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762725.098099:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762725.098102:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f174487 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.098104:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f17448e lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.098111:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762725.098115:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f17448e lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098116:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f174487 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.098122:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f17448e lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098127:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7:0x0].0x0 (ffff9c37d3bee700) 00010000:00010000:2.0:1789762725.098129:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762725.098133:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f174487 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.098137:0:8069:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7:0x0].0x0 (ffff9c37d3bee700) 00010000:00010000:3.0:1789762725.098139:0:8069:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789762725.098140:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cee00/0xdf5a7ac59f174487 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.098143:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f17448e lrc: 1/0,0 mode: --/PW res: [0x240000402:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.098149:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf600/0xdf5a7ac59f17448e lrc: 0/0,0 mode: --/PW res: [0x240000402:0x7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098154:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174479 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098161:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174479 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.098161:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762725.098167:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174479 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.098168:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762725.098170:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762725.098173:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cf200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762725.098177:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174479 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098184:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174479 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098192:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:2.0:1789762725.098194:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762725.098198:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174479 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098215:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cf200/0xdf5a7ac59f174479 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762725.098223:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004da22949 reply state 00000000f727645f transno 4294967355 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174480 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098231:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174480 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.098235:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x73550ecb (ffff9c37d3bee000) 00010000:00010000:2.0:1789762725.098237:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762725.098239:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762725.098272:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:12157:x1876702067264128:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1593us (1678us total) trans 4294967355 rc 0/0 00000100:00100000:2.0:1789762725.098277:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 133 00000100:00100000:2.0:1789762725.100319:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.100349:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067264384 00000100:00100000:3.0:1789762725.100355:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067264384 00000100:00100000:3.0:1789762725.100382:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 134 00000100:00100000:3.0:1789762725.100388:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b8c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:12157:x1876702067264384:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:3.0:1789762725.100419:0:13389:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1789762725.100431:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cfe00/0xdf5a7ac59f1743d1 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c457 expref: 21 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100444:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f1742ff lrc: 0/0,0 mode: --/CR res: [0x240000402:0x8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c403 expref: 20 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100451:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x8:0x0].0x0 (ffff9c37c72ba900) 00010000:00010000:3.0:1789762725.100457:0:13389:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762725.100465:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935200/0xdf5a7ac59f174495 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100471:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935200/0xdf5a7ac59f174495 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100477:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935200/0xdf5a7ac59f174495 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100484:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934000/0xdf5a7ac59f17449c lrc: 3/0,1 mode: --/PW res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100489:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934000/0xdf5a7ac59f17449c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100493:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934000/0xdf5a7ac59f17449c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100536:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f1744a3 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100542:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f1744a3 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100547:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f1744a3 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789762725.100842:0:13389:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].5, 64 off8512 00000040:00080000:3.0:1789762725.100853:0:13389:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].5, 64 off8512 00010000:00010000:3.0:1789762725.100885:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f1744aa lrc: 3/0,1 mode: --/PW res: [0x240000402:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100892:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f1744aa lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100898:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f1744aa lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100903:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f1744aa lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100907:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f1744aa lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100911:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f1744aa lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100943:0:13389:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762725.100952:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f1744a3 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100962:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f1744a3 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100966:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f1744a3 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100970:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762725.100972:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f1744a3 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100976:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f1744a3 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100983:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x8:0x0].0x0 (ffff9c37c72ba600) 00010000:00010000:3.0:1789762725.100985:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789762725.100987:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f1744a3 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.100991:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935400/0xdf5a7ac59f1744a3 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.100999:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762725.101002:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935200/0xdf5a7ac59f174495 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.101008:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f1744aa lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.101009:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935200/0xdf5a7ac59f174495 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.101026:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935200/0xdf5a7ac59f174495 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.101028:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762725.101033:0:6408:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f1744aa lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.101034:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762725.101041:0:6408:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f1744aa lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.101043:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935200/0xdf5a7ac59f174495 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.101059:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935200/0xdf5a7ac59f174495 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.101061:0:6408:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x8:0x0].0x0 (ffff9c37c72ba600) 00010000:00010000:0.0:1789762725.101065:0:6408:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762725.101068:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f1744aa lrc: 1/0,0 mode: --/PW res: [0x240000402:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.101068:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:3.0:1789762725.101074:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762725.101078:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935200/0xdf5a7ac59f174495 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.101079:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f1744aa lrc: 0/0,0 mode: --/PW res: [0x240000402:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.101094:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935200/0xdf5a7ac59f174495 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.101109:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00010000:3.0:1789762725.101109:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000048fc4114 reply state 00000000a69b0851 transno 4294967356 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934000/0xdf5a7ac59f17449c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.101112:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762725.101115:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762725.101121:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934000/0xdf5a7ac59f17449c lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.101126:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x2b0e463d (ffff9c37c72bae00) 00010000:00010000:3.0:1789762725.101127:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1789762725.101129:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762725.101162:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b8c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:12157:x1876702067264384:12345-192.168.203.37@tcp:36:rm.0 Request processed in 773us (844us total) trans 4294967356 rc 0/0 00000100:00100000:3.0:1789762725.101167:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 134 00000100:00100000:0.0:1789762725.102173:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.102198:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.102205:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067264512 00000100:00100000:3.0:1789762725.102211:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067264512 00000100:00100000:2.0:1789762725.102228:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067264768 00000100:00100000:2.0:1789762725.102234:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067264768 00000100:00100000:3.0:1789762725.102247:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 17 00000100:00100000:3.0:1789762725.102257:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067264512:12345-192.168.203.37@tcp:103:rm.0 00010000:00010000:3.0:1789762725.102264:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1789762725.102269:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 33 00000100:00100000:2.0:1789762725.102282:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:12157:x1876702067264768:12345-192.168.203.37@tcp:35:rm.0 00010000:00010000:3.0:1789762725.102290:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f17420a lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3b6 expref: 7 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.102301:0:6404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743700) 00010000:00010000:3.0:1789762725.102308:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762725.102337:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067264512:12345-192.168.203.37@tcp:103:rm.0 Request processed in 83us (165us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762725.102345:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 17 00000100:00100000:2.0:1789762725.102377:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:12157:x1876702067264768:12345-192.168.203.37@tcp:35:rm.0 Request processed in 95us (175us total) trans 4294967357 rc 0/0 00000100:00100000:2.0:1789762725.102383:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 33 00000100:00100000:0.0:1789762725.103040:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.103062:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067265024 00000100:00100000:3.0:1789762725.103067:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067265024 00000100:00100000:3.0:1789762725.103091:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 135 00000100:00100000:3.0:1789762725.103097:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:12157:x1876702067265024:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:3.0:1789762725.103115:0:6418:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1789762725.103124:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f17446b lrc: 0/0,0 mode: --/CR res: [0x240000402:0x6:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4b9 expref: 22 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.103135:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f17428f lrc: 0/0,0 mode: --/PR res: [0x240000402:0x6:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3e7 expref: 21 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.103141:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:3.0:1789762725.103145:0:6418:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762725.103181:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.103186:0:6418:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.103193:0:6418:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762725.103214:0:6418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffa300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6418:1876702074612480:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1789762725.103245:0:6418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762725.103278:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074612480 00000100:00100000:0.0:1789762725.103282:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074612480 00000100:00100000:3.0:1789762725.103287:0:6418:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d24bec00 going to sleep for 11 seconds 00000100:00100000:0.0:1789762725.103316:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:0.0:1789762725.103323:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6418:x1876702074612480:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1789762725.103330:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762725.103345:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f1744b8 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f1744b1 expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.103360:0:6424:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cec00/0xdf5a7ac59f17421f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3c4 expref: 5 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.103367:0:6424:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f1744b8 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f1744b1 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.103384:0:6424:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cec00/0xdf5a7ac59f17421f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3c4 expref: 5 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.103393:0:6424:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cec00/0xdf5a7ac59f17421f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c3c4 expref: 5 pid: 6418 timeout: 298 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762725.103400:0:6424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d98a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074612608:192.168.203.37@tcp:104: 00000100:00100000:0.0:1789762725.103460:0:6424:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff66e480 going to sleep for 11 seconds 00000100:00100000:3.0:1789762725.104246:0:6424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d98a5f80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074612608:192.168.203.37@tcp:104: 00010000:00010000:3.0:1789762725.104263:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f1744b8 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1744b1 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.104270:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762725.104271:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762725.104274:0:6424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f1744b8 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1744b1 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.104280:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa7200, rc 0) 00000100:00100000:3.0:1789762725.104308:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6418:x1876702074612480:12345-0@lo:101:mdt00_000.0 Request processed in 986us (1062us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762725.104312:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:2.0:1789762725.104380:0:6418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffa300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6418:1876702074612480:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1789762725.104392:0:6418:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.104429:0:6418:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762725.104576:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.104806:0:6402:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074612736 00000100:00100000:3.0:1789762725.104811:0:6402:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074612736 00000100:00100000:3.0:1789762725.104837:0:6402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 39 00000100:00100000:3.0:1789762725.104842:0:6402:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9878:x1876702074612736:12345-0@lo:105: 00010000:00010000:3.0:1789762725.104848:0:6402:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.104855:0:6402:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.104870:0:6402:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.104875:0:6402:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.104880:0:6402:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.104887:0:6402:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.104900:0:6402:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c37ff935a00) 00010000:00010000:2.0:1789762725.104906:0:6418:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.104943:0:6418:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762725.104953:0:6402:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9878:x1876702074612736:12345-0@lo:105: Request processed in 109us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762725.104960:0:6402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 39 00000004:00080000:2.0:1789762725.104961:0:6418:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1789762725.104996:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.105003:0:6418:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.105010:0:6418:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762725.105021:0:6418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9fff800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6418:1876702074612864:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1789762725.105053:0:6418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762725.105079:0:6418:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7fbe9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762725.105090:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074612864 00000100:00100000:3.0:1789762725.105097:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074612864 00000100:00100000:2.0:1789762725.105150:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:2.0:1789762725.105158:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6418:x1876702074612864:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1789762725.105166:0:6423:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762725.105176:0:6423:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf000/0xdf5a7ac59f1744c6 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f1744bf expref: -99 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.105199:0:6423:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf000/0xdf5a7ac59f1744c6 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f1744bf expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.105207:0:6423:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf000/0xdf5a7ac59f1744c6 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1744bf expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.105212:0:6423:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff9c37d3bee600) 00010000:00010000:2.0:1789762725.105215:0:6423:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762725.105216:0:6423:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8cf000, rc 0) 00000100:00100000:2.0:1789762725.105245:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6418:x1876702074612864:12345-0@lo:101:mdt00_000.0 Request processed in 89us (197us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762725.105251:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00100000:3.0:1789762725.105270:0:6418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9fff800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6418:1876702074612864:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1789762725.105282:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1744c6 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.105289:0:6418:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1744c6 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762725.105299:0:6418:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1789762725.105327:0:6418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ff8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6418:1876702074612992:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1789762725.105348:0:6418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762725.105364:0:6418:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d24be600 going to sleep for 11 seconds 00000100:00100000:3.0:1789762725.105491:0:6418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ff8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6418:1876702074612992:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1789762725.105520:0:6418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ff8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6418:1876702074613120:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1789762725.105538:0:6418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762725.105552:0:6418:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d24be600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762725.105575:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074613120 00000100:00100000:2.0:1789762725.105580:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074613120 00000100:00100000:2.0:1789762725.105603:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1789762725.105608:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6418:x1876702074613120:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1789762725.105727:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6418:x1876702074613120:12345-0@lo:1000:mdt00_000.0 Request processed in 117us (186us total) trans 0 rc 1/1 00000100:00100000:3.0:1789762725.105734:0:6418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ff8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6418:1876702074613120:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1789762725.105735:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00010000:00010000:3.0:1789762725.105768:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 3/0,1 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.105773:0:6418:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 3/0,1 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.105778:0:6418:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 3/0,1 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762725.105785:0:6418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ff8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6418:1876702074613248:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1789762725.105801:0:6418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762725.105815:0:6418:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d24be9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789762725.105832:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074613248 00000100:00100000:0.0:1789762725.105837:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074613248 00000100:00100000:0.0:1789762725.105861:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:0.0:1789762725.105866:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6418:x1876702074613248:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1789762725.105873:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762725.105881:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f1744d4 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f1744cd expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.105893:0:6424:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f1744d4 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f1744cd expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.105910:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f1744d4 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1744cd expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.105948:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37e125ac00) 00010000:00010000:0.0:1789762725.105951:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762725.105954:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa7800, rc 0) 00000100:00100000:0.0:1789762725.105986:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6418:x1876702074613248:12345-0@lo:101:mdt00_000.0 Request processed in 120us (184us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762725.105992:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:3.0:1789762725.106004:0:6418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ff8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6418:1876702074613248:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1789762725.106019:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1744d4 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.106030:0:6418:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1744d4 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.106047:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f1744db lrc: 3/0,1 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.106058:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f1744db lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.106066:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f1744db lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789762725.106515:0:6418:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].4, 424 off35584 00000040:00080000:3.0:1789762725.106548:0:6418:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].4, 504 off36048 00000004:00080000:3.0:1789762725.106553:0:6418:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37d24be600 with version 9 00000004:00080000:0.0:1789762725.106596:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 9 rpc_version 9 00000020:00100000:0.0:1789762725.106615:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762725.106621:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37d98a6300 x1876702074613504/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1744/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762725.106633:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d98a6300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074613504:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762725.106656:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762725.106673:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff66e480 going to sleep for 11 seconds 00000100:00100000:3.0:1789762725.106676:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074613504 00000100:00100000:3.0:1789762725.106681:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074613504 00000100:00100000:3.0:1789762725.106704:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:3.0:1789762725.106709:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7224:x1876702074613504:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1789762725.107082:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7224:x1876702074613504:12345-0@lo:1000:osp_up0-1.0 Request processed in 370us (422us total) trans 4294967340 rc 0/0 00000100:00100000:3.0:1789762725.107090:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:0.0:1789762725.107153:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d98a6300 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074613504:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1789762725.107165:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x6:0x0]: returning size 16384 00010000:00010000:3.0:1789762725.107174:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f1744db lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107179:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f1744db lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107184:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f1744db lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107187:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762725.107189:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f1744db lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107202:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f1744db lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107209:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37c72ba900) 00010000:00010000:3.0:1789762725.107211:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762725.107213:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f1744db lrc: 2/0,0 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107217:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f1744db lrc: 0/0,0 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107227:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1744d4 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107232:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f1744d4 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107237:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f1744d4 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107249:0:6418:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1789762725.107254:0:6418:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1789762725.107257:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107262:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107267:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.107269:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762725.107275:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f1744d4 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.107284:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934e00 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:3.0:1789762725.107285:0:6418:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762725.107289:0:7217:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f1744d4 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762725.107290:0:6418:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1789762725.107298:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1744c6 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762725.107300:0:7217:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f1744d4 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.107305:0:6418:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1744c6 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.107340:0:7217:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 2/0,0 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f1744d4 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.107345:0:7217:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762725.107349:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1789762725.107357:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:12157:x1876702067265024:12345-192.168.203.37@tcp:36:rm.0 Request processed in 4257us (4315us total) trans 4294967358 rc 0/0 00010000:00010000:2.0:1789762725.107358:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.107364:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 1/0,0 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f1744d4 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762725.107372:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 135 00010000:00010000:2.0:1789762725.107381:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935a00 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:0.0:1789762725.107383:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff934e00/0xdf5a7ac59f1744cd lrc: 0/0,0 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f1744d4 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.107387:0:6407:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.107397:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762725.107400:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762725.107417:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00010000:2.0:1789762725.107425:0:6407:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762725.107434:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d98a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074613632:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762725.107449:0:6407:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.107458:0:6407:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762725.107470:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762725.107481:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789762725.107482:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ff935a00/0xdf5a7ac59f1744b1 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f1744b8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.107502:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762725.107506:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762725.107507:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074613632 00010000:00010000:2.0:1789762725.107510:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762725.107512:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074613632 00000100:00100000:2.0:1789762725.107568:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074613760 00000100:00100000:2.0:1789762725.107573:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074613760 00000100:00100000:3.0:1789762725.107598:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 18 00000100:00100000:0.0:1789762725.107614:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 19 00000100:00100000:3.0:1789762725.107615:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3426:x1876702074613632:12345-0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762725.107625:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3427:x1876702074613760:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789762725.107628:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762725.107635:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762725.107645:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f1744d4 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x6:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f1744cd expref: 11 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.107649:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f1744b8 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f1744b1 expref: 11 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.107670:0:6405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762725.107673:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762725.107673:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x0 (ffff9c37e125ac00) 00010000:00010000:3.0:1789762725.107684:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762725.107705:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3427:x1876702074613760:12345-0@lo:103:ldlm_bl.0 Request processed in 83us (172us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762725.107712:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 19 00000100:00100000:3.0:1789762725.107779:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3426:x1876702074613632:12345-0@lo:103:ldlm_bl.0 Request processed in 172us (299us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762725.107785:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 18 00000100:00100000:0.0:1789762725.107808:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d98a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074613632:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762725.319989:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.320021:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.320044:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.320063:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.320080:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.320103:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.320104:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067265664 00000100:00100000:2.0:1789762725.320109:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067265280 00000100:00100000:3.0:1789762725.320113:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067265664 00000100:00100000:2.0:1789762725.320116:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067265280 00000100:00100000:0.0:1789762725.320143:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762725.320164:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 303 00000100:00100000:3.0:1789762725.320173:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067265792 00000100:00100000:2.0:1789762725.320176:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1553450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067265280:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762725.320182:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067265792 00000100:00100000:0.0:1789762725.320187:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067266048 00000100:00100000:0.0:1789762725.320192:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067266048 00000100:00100000:0.0:1789762725.320245:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 210 00000100:00100000:2.0:1789762725.320247:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1553450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067265280:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 67us (217us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762725.320257:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:2396:x1876702067265536:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762725.320268:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 303 00000100:00100000:0.0:1789762725.320308:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:2396:x1876702067265536:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 54us (243us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762725.320318:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 211 00000100:00100000:0.0:1789762725.320329:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 210 00000100:00100000:2.0:1789762725.320335:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067265920:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762725.320340:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 81 00000100:00100000:0.0:1789762725.320350:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 212 00000100:00100000:3.0:1789762725.320357:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2395:x1876702067265664:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762725.320362:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067266048:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762725.320388:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067265920:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 56us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762725.320394:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067266048:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 34us (249us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762725.320423:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 212 00000100:00100000:3.0:1789762725.320452:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2395:x1876702067265664:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 96us (403us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762725.320465:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 81 00000100:00100000:2.0:1789762725.320526:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 211 00000100:00100000:2.0:1789762725.378573:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067266176 00000100:00100000:2.0:1789762725.378583:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067266176 00000100:00100000:2.0:1789762725.378633:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 136 00000100:00100000:2.0:1789762725.378645:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8abb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:12168:x1876702067266176:12345-192.168.203.37@tcp:101:mkdir.0 00010000:00010000:2.0:1789762725.378663:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762725.378680:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1744e2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c4c0 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.378733:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1744e9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.378748:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1744e9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.378761:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1744e9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762725.378796:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762725.378799:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762725.378815:0:6419:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1744e9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.378834:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1744e2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4c0 expref: 7 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.378848:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1744e9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4c0 expref: 6 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.378862:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762725.378865:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762725.378867:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8ce000, rc 0) 00000100:00100000:2.0:1789762725.378914:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8abb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:12168:x1876702067266176:12345-192.168.203.37@tcp:101:mkdir.0 Request processed in 273us (399us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762725.378941:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 136 00000100:00100000:0.0:1789762725.379954:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.380258:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762725.380294:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067266560 00000100:00100000:2.0:1789762725.380300:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067266560 00000100:00100000:2.0:1789762725.380334:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 138 00000100:00100000:2.0:1789762725.380344:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:12168:x1876702067266560:12345-192.168.203.37@tcp:36:mkdir.0 00010000:00010000:2.0:1789762725.380372:0:13389:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762725.380385:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1744e9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4c0 expref: 6 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.380395:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762725.380398:0:13389:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762725.380458:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf400/0xdf5a7ac59f1744f0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.380465:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf400/0xdf5a7ac59f1744f0 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.380471:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf400/0xdf5a7ac59f1744f0 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.380478:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.380494:0:13389:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf000/0xdf5a7ac59f1744c6 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1744bf expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.380499:0:13389:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 4/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.380525:0:13389:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf000/0xdf5a7ac59f1744c6 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xdf5a7ac59f1744bf expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762725.380537:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffe680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:13389:1876702074613888:0@lo:104: 00000100:00100000:2.0:1789762725.380572:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762725.380625:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff872540 going to sleep for 11 seconds 00010000:00010000:0.0:1789762725.380738:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762725.380746:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f1744c6 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762725.380748:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffe680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:13389:1876702074613888:0@lo:104: 00010000:00010000:0.0:1789762725.380771:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cee00 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:2.0:1789762725.380775:0:13389:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.380777:0:8069:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xdf5a7ac59f1744c6 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762725.380797:0:8069:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xdf5a7ac59f1744c6 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.380817:0:8069:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f1744c6 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.380823:0:8069:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1789762725.380844:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f1744c6 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.380850:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ee8cee00/0xdf5a7ac59f1744bf lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f1744c6 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762725.380861:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d19f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074614016:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762725.380866:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762725.380869:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762725.380872:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762725.380890:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762725.380941:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074614016 00000100:00100000:0.0:1789762725.380945:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074614016 00010000:00010000:0.0:1789762725.380961:0:6406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf000/0xdf5a7ac59f1744c6 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xdf5a7ac59f1744bf expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762725.380982:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 59 00000100:00100000:0.0:1789762725.380988:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffe680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074614016:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762725.381003:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1789762725.390508:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967340 is committed 00010000:00010000:0.0:1789762725.392201:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf000/0xdf5a7ac59f1744c6 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xdf5a7ac59f1744bf expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.392217:0:6406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff9c37d3bee600) 00010000:00010000:0.0:1789762725.392221:0:6406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.392228:0:6406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.392233:0:6406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.392254:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762725.392299:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffe680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074614016:12345-0@lo:103:ldlm_bl.0 Request processed in 11309us (11406us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762725.392308:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 59 00000100:00100000:3.0:1789762725.392327:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d19f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074614016:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762725.398977:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067266688 00000100:00100000:0.0:1789762725.398985:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067266688 00000100:00100000:0.0:1789762725.399044:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 139 00000100:00100000:0.0:1789762725.399054:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb931880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:12169:x1876702067266688:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762725.399074:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762725.399090:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174505 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c4c7 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.399148:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f17450c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.399174:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f17450c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.399186:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f17450c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762725.399213:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762725.399217:0:7744:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762725.399237:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f17450c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.399262:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174505 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4c7 expref: 9 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.399278:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f17450c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4c7 expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.399288:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762725.399291:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762725.399295:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa6c00, rc 0) 00000100:00100000:0.0:1789762725.399358:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb931880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:12169:x1876702067266688:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 305us (554us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762725.399367:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 139 00000100:00100000:0.0:1789762725.401222:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.402914:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067267072 00000100:00100000:0.0:1789762725.402949:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067267072 00000100:00100000:0.0:1789762725.402990:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 141 00000100:00100000:0.0:1789762725.403001:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:12169:x1876702067267072:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762725.403020:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762725.403033:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6600/0xdf5a7ac59f17452f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c4d5 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.403073:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174536 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.403084:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174536 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.403095:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174536 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762725.403116:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762725.403120:0:11692:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762725.403135:0:11692:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174536 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.403155:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6600/0xdf5a7ac59f17452f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4d5 expref: 9 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.403171:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174536 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4d5 expref: 8 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.403179:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762725.403182:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762725.403185:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa7600, rc 0) 00000100:00100000:0.0:1789762725.403224:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:12169:x1876702067267072:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 225us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762725.403233:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 141 00000100:00100000:0.0:1789762725.404132:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.405544:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067267584 00000100:00100000:3.0:1789762725.405550:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067267584 00000100:00100000:3.0:1789762725.405582:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 143 00000100:00100000:3.0:1789762725.405591:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:12169:x1876702067267584:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:3.0:1789762725.405627:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174559 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.405636:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174559 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.405643:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174559 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762725.405775:0:6418:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b3eeebe1 reply state 00000000ef73cdc6 transno 4294967343 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174559 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.405789:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174559 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.405798:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:3.0:1789762725.405809:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:3.0:1789762725.405813:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762725.405820:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x9:0x0]: returning size 16384 00000100:00100000:3.0:1789762725.405866:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:12169:x1876702067267584:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 276us (351us total) trans 4294967343 rc 0/0 00000100:00100000:3.0:1789762725.405873:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 143 00000100:00100000:0.0:1789762725.411371:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762725.413087:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067268224 00000100:00100000:2.0:1789762725.413094:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067268224 00000100:00100000:2.0:1789762725.413156:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 145 00000100:00100000:2.0:1789762725.413171:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:12170:x1876702067268224:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:2.0:1789762725.413203:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762725.413222:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174575 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4e3 expref: 13 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.413254:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:2.0:1789762725.413259:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762725.413273:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f17457c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.413284:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f17457c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.413294:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f17457c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762725.413462:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000af070ac8 reply state 0000000077cb1467 transno 4294967345 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f17457c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.413472:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f17457c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.413477:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:2.0:1789762725.413479:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:2.0:1789762725.413481:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762725.413486:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x9:0x0]: returning size 16384 00000100:00100000:2.0:1789762725.413530:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:12170:x1876702067268224:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 365us (502us total) trans 4294967345 rc 0/0 00000100:00100000:2.0:1789762725.413539:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 145 00000100:00100000:2.0:1789762725.418316:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.419802:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762725.419835:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067268608 00000100:00100000:2.0:1789762725.419840:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067268608 00000100:00100000:2.0:1789762725.419870:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 147 00000100:00100000:2.0:1789762725.419876:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:12171:x1876702067268608:12345-192.168.203.37@tcp:101:touch.500 00010000:00010000:2.0:1789762725.419891:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762725.419899:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfc00/0xdf5a7ac59f17459f lrc: 2/0,0 mode: --/CW res: [0x200000402:0x9:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c4f1 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420026:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1745a6 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x9:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420035:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1745a6 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420041:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1745a6 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420085:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1745ad lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420090:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1745ad lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420095:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1745ad lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762725.420487:0:11692:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x4:0x0], index 0: 0 00010000:00010000:2.0:1789762725.420607:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf400/0xdf5a7ac59f1745b4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420614:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf400/0xdf5a7ac59f1745b4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420620:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf400/0xdf5a7ac59f1745b4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420650:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1745a6 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420655:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1745a6 lrc: 2/0,0 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420659:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1745a6 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420662:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8ce600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762725.420664:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1745a6 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420668:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1745a6 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420674:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:2.0:1789762725.420676:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762725.420679:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1745a6 lrc: 2/0,0 mode: --/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420683:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f1745a6 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762725.420689:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003f083e54 reply state 0000000017eea781 transno 4294967346 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1745ad lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420695:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1745ad lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420699:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x4009f790 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762725.420705:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762725.420707:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762725.420724:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfc00/0xdf5a7ac59f17459f lrc: 0/0,0 mode: --/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4f1 expref: 18 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420733:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cf400/0xdf5a7ac59f1745b4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4f1 expref: 17 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.420738:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37d3bee700) 00010000:00010000:2.0:1789762725.420739:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762725.420741:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8cf400, rc 0) 00000100:00100000:2.0:1789762725.420768:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:12171:x1876702067268608:12345-192.168.203.37@tcp:101:touch.500 Request processed in 893us (970us total) trans 4294967346 rc 0/0 00000100:00100000:2.0:1789762725.420773:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 147 00000100:00100000:2.0:1789762725.421695:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.422682:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067268992 00000100:00100000:3.0:1789762725.422688:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067268992 00000100:00100000:3.0:1789762725.422714:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 149 00000100:00100000:3.0:1789762725.422720:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f63af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:12171:x1876702067268992:12345-192.168.203.37@tcp:101:touch.500 00010000:00010000:3.0:1789762725.422729:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762725.422737:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935200/0xdf5a7ac59f1745c9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c4f8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.422760:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935200/0xdf5a7ac59f1745c9 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4f8 expref: 20 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.422769:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935200/0xdf5a7ac59f1745c9 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4f8 expref: 20 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.422781:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37db743c00) 00010000:00010000:3.0:1789762725.422784:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789762725.422787:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff935200, rc 0) 00000100:00100000:3.0:1789762725.422822:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f63af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:12171:x1876702067268992:12345-192.168.203.37@tcp:101:touch.500 Request processed in 101us (168us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762725.422832:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 149 00000100:00100000:0.0:1789762725.423866:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762725.423961:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067269120 00000100:00100000:2.0:1789762725.423967:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067269120 00000100:00100000:2.0:1789762725.424009:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 91 00000100:00100000:2.0:1789762725.424020:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2397:x1876702067269120:12345-192.168.203.37@tcp:2:touch.500 00000100:00100000:2.0:1789762725.424181:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:2397:x1876702067269120:12345-192.168.203.37@tcp:2:touch.500 Request processed in 165us (315us total) trans 4294967450 rc 0/0 00000100:00100000:2.0:1789762725.424188:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 91 00000100:00100000:2.0:1789762725.425012:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.425044:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067269376 00000100:00100000:3.0:1789762725.425048:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067269376 00000100:00100000:3.0:1789762725.425079:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 35 00000100:00100000:3.0:1789762725.425088:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:12171:x1876702067269376:12345-192.168.203.37@tcp:35:touch.500 00000100:00100000:3.0:1789762725.425243:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:12171:x1876702067269376:12345-192.168.203.37@tcp:35:touch.500 Request processed in 156us (232us total) trans 4294967348 rc 0/0 00000100:00100000:3.0:1789762725.425249:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 35 00000100:00100000:3.0:1789762725.429242:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.430660:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.430690:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067269632 00000100:00100000:3.0:1789762725.430694:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067269632 00000100:00100000:3.0:1789762725.430720:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 92 00000100:00100000:3.0:1789762725.430726:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:2397:x1876702067269632:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:3.0:1789762725.430733:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762725.430741:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1745ec lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c506 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762725.430754:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 0, mtime: 0x6aad9ca5, atime: 0x6aad9ca5, ctime: 0x6aad9ca5, blocks: 0x1 00010000:00010000:3.0:1789762725.430764:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=198s v=(0 0) 00010000:00010000:3.0:1789762725.430768:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762725.430774:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1745ec lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c506 expref: 44 pid: 8054 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.430786:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1745ec lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c506 expref: 44 pid: 8054 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.430796:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff935400, rc 0) 00000100:00100000:3.0:1789762725.430830:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+44:2397:x1876702067269632:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 103us (169us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762725.430846:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 92 00000100:00100000:2.0:1789762725.431740:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.431772:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067269888 00000100:00100000:0.0:1789762725.431777:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067269888 00000100:00100000:0.0:1789762725.431803:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 151 00000100:00100000:0.0:1789762725.431809:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:12172:x1876702067269888:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:0.0:1789762725.431828:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762725.431839:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f1745e5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4ff expref: 22 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.431849:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37db743c00) 00010000:00010000:0.0:1789762725.431851:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762725.431861:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f1745f3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.431869:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f1745f3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.431874:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f1745f3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762725.431973:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008111d165 reply state 000000002a21071d transno 4294967349 ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f1745f3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.431981:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f1745f3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.431995:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37db743c00) 00010000:00010000:0.0:1789762725.431997:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:0.0:1789762725.431999:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762725.432043:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:12172:x1876702067269888:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 232us (302us total) trans 4294967349 rc 0/0 00000100:00100000:0.0:1789762725.432051:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 151 00000100:00100000:2.0:1789762725.432671:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067270016 00000100:00100000:2.0:1789762725.432675:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067270016 00000100:00100000:2.0:1789762725.432702:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 93 00000100:00100000:2.0:1789762725.432710:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee8c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+44:2398:x1876702067270016:12345-192.168.203.37@tcp:2:chmod.0 00002000:00010000:2.0:1789762725.432793:0:9037:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1789762725.432796:0:9037:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb mtime from disk: 1789762725 -> 1789762725 00002000:00010000:2.0:1789762725.432798:0:9037:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb atime from disk: 1789762725 -> 1789762725 00002000:00010000:2.0:1789762725.432799:0:9037:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb ctime from disk: 1789762725 -> 1789762725 00002000:00010000:2.0:1789762725.432801:0:9037:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1789762725.432827:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee8c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+45:2398:x1876702067270016:12345-192.168.203.37@tcp:2:chmod.0 Request processed in 118us (199us total) trans 4294967451 rc 0/0 00000100:00100000:2.0:1789762725.432831:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 93 00000100:00100000:0.0:1789762725.451308:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067270272 00000100:00100000:0.0:1789762725.451314:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067270272 00000100:00100000:0.0:1789762725.451343:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 152 00000100:00100000:0.0:1789762725.451350:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b8f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:12178:x1876702067270272:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:0.0:1789762725.451361:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762725.451370:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f1745fa lrc: 2/0,0 mode: --/CW res: [0x200000402:0x9:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c50d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451447:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174601 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x9:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451455:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174601 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451462:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174601 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451468:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174608 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451472:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174608 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451486:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174608 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451525:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f17460f lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451531:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f17460f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451536:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f17460f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762725.451541:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb:0x0]: returning size 0 00010000:00010000:0.0:1789762725.451580:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174601 lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451586:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174601 lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451590:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174601 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451594:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174601 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451598:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762725.451600:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174601 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451604:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174601 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451619:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:0.0:1789762725.451621:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762725.451623:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174601 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451628:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174601 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451635:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174608 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451638:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174608 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451642:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174608 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451646:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762725.451648:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174608 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451652:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174608 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451656:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x4009f790 (ffff9c37d3bee500) 00010000:00010000:0.0:1789762725.451658:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762725.451659:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174608 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451663:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174608 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451690:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f1745fa lrc: 0/0,0 mode: --/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c50d expref: 26 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451699:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f17460f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c50d expref: 25 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.451704:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37db743c00) 00010000:00010000:0.0:1789762725.451706:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762725.451707:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa6800, rc 0) 00000100:00100000:0.0:1789762725.451741:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b8f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:12178:x1876702067270272:12345-192.168.203.37@tcp:101:dd.500 Request processed in 390us (514us total) trans 4294967350 rc 0/0 00000100:00100000:0.0:1789762725.451748:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 152 00000100:00100000:2.0:1789762725.453032:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067270528 00000100:00100000:2.0:1789762725.453038:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067270528 00000100:00100000:2.0:1789762725.453064:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 153 00000100:00100000:2.0:1789762725.453071:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:12178:x1876702067270528:12345-192.168.203.37@tcp:36:dd.500 00010000:00010000:2.0:1789762725.453099:0:11692:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762725.453109:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f17460f lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c50d expref: 25 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.453119:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37db743c00) 00010000:00010000:2.0:1789762725.453122:0:11692:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762725.453200:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f17461d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.453208:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f17461d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.453214:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f17461d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762725.453241:0:11692:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967351 00000004:00010000:2.0:1789762725.453264:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000705ebc69 reply state 00000000d1fb3417 transno 4294967352 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f17461d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.453272:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce600/0xdf5a7ac59f17461d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.453276:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37db743c00) 00010000:00010000:2.0:1789762725.453278:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762725.453280:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762725.453320:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:12178:x1876702067270528:12345-192.168.203.37@tcp:36:dd.500 Request processed in 250us (337us total) trans 4294967352 rc 0/0 00000100:00100000:2.0:1789762725.453325:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 153 00000020:00080000:2.0:1789762725.453328:0:11692:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789762725.454071:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067270656 00000100:00100000:2.0:1789762725.454075:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067270656 00000100:00100000:2.0:1789762725.454099:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 94 00000100:00100000:2.0:1789762725.454105:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+45:12178:x1876702067270656:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:2.0:1789762725.454111:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762725.454118:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174624 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c514 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.454133:0:8053:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1745ec lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c506 expref: 46 pid: 8054 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.454139:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762725.454142:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762725.454145:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174624 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000080000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c514 expref: 46 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.454176:0:8053:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1745ec lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c506 expref: 46 pid: 8054 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.454187:0:8053:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1745ec lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c506 expref: 46 pid: 8054 timeout: 298 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762725.454197:0:8053:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9fff100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074614144:192.168.203.37@tcp:104: 00000100:00100000:2.0:1789762725.454243:0:8053:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c9fc8000 going to sleep for 11 seconds 00000100:00100000:2.0:1789762725.454976:0:8053:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9fff100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074614144:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762725.454998:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174624 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c514 expref: 46 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.455010:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee8cee00, rc 0) 00000100:00100000:2.0:1789762725.455043:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+46:12178:x1876702067270656:12345-192.168.203.37@tcp:101:dd.500 Request processed in 936us (1010us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762725.455051:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 94 00000100:00100000:3.0:1789762725.455113:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762725.455153:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067270784 00000100:00100000:2.0:1789762725.455158:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067270784 00010000:00010000:2.0:1789762725.455179:0:6404:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1745ec lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c506 expref: 46 pid: 8054 timeout: 298 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762725.455207:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 60 00000100:00100000:2.0:1789762725.455216:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+46:2398:x1876702067270784:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762725.455236:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762725.455242:0:6404:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1745ec lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c506 expref: 46 pid: 8054 timeout: 298 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.455256:0:6404:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1745ec lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c506 expref: 46 pid: 8054 timeout: 298 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.455271:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1745ec lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c506 expref: 46 pid: 8054 timeout: 298 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.455280:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762725.455282:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762725.455286:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174624 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c514 expref: 45 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.455292:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174624 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c514 expref: 45 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.455309:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174624 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c514 expref: 45 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762725.455316:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9fff100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074614272:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762725.455347:0:6404:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c9fc8000 going to sleep for 11 seconds 00000100:00100000:2.0:1789762725.456031:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9fff100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074614272:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762725.456043:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762725.456087:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+45:2398:x1876702067270784:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 872us (973us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762725.456096:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 60 00000100:00100000:3.0:1789762725.456138:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.456170:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067270912 00000100:00100000:0.0:1789762725.456176:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067270912 00002000:00010000:0.0:1789762725.456196:0:14171:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/4 (0->18446744073709551615) 00002000:00100000:0.0:1789762725.456200:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36c9ffb100 with x1876702067270912 ext(0->18446744073709551615) 00002000:00010000:0.0:1789762725.456206:0:14171:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36c9ffb100. 00000100:00100000:0.0:1789762725.456216:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 198185749972 time 198185861375 state 0x0 ref 1 00000100:00100000:0.0:1789762725.456224:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 198385861375 time 198185861375 state 0x1 ref 1 00000100:00100000:0.0:1789762725.456244:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 1 deadline 198385861375 time 198703324351 state 0x0 ref 1 00000100:00100000:0.0:1789762725.456253:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+45:2397:x1876702067270912:12345-192.168.203.37@tcp:10:dd.500 00002000:00010000:0.0:1789762725.456369:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1789762725.456372:0:14171:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb mtime from disk: 1789762725 -> 1789762725 00002000:00010000:0.0:1789762725.456379:0:14171:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb atime from disk: 1789762725 -> 1789762725 00002000:00010000:0.0:1789762725.456381:0:14171:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb ctime from disk: 1789762725 -> 1789762725 00002000:00010000:0.0:1789762725.456382:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:0.0:1789762725.456430:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+46:2397:x1876702067270912:12345-192.168.203.37@tcp:10:dd.500 Request processed in 177us (294us total) trans 4294967452 rc 0/0 00000100:00100000:0.0:1789762725.456436:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 34 00000100:00100000:0.0:1789762725.456440:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 1 deadline 198385861375 time 198703324351 state 0x2 ref 0 00002000:00010000:0.0:1789762725.456444:0:14171:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/4 (0->18446744073709551615) 00002000:00100000:0.0:1789762725.456446:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36c9ffb100 with x1876702067270912 ext(0->18446744073709551615) 00002000:00010000:0.0:1789762725.456450:0:14171:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36c9ffb100. 00000100:00100000:3.0:1789762725.459068:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762725.460031:0:8059:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6094380 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0000_UUID:8059:1876702074614400:192.168.203.37@tcp:104: 00010000:00010000:3.0:1789762725.460047:0:8059:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f17462b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762725.460183:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762725.460256:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067271168 00000100:00100000:0.0:1789762725.460262:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067271168 00010000:00010000:0.0:1789762725.460280:0:6405:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174624 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000480020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c514 expref: 47 pid: 8053 timeout: 298 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762725.460305:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 61 00000100:00100000:0.0:1789762725.460312:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9fff100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+47:2397:x1876702067271168:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762725.460319:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762725.460328:0:6405:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174624 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000480020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c514 expref: 47 pid: 8053 timeout: 298 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.460337:0:6405:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174624 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000480020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c514 expref: 47 pid: 8053 timeout: 298 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.460347:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174624 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801480020020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c514 expref: 47 pid: 8053 timeout: 298 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.460362:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762725.460365:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762725.460368:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f17462b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.460374:0:6405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f17462b lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762725.460388:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762725.460397:0:8059:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f17462b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.460420:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f17462b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762725.460432:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9fff100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+46:2397:x1876702067271168:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 120us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762725.460439:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 61 00000001:00080000:3.0:1789762725.487648:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967452 is committed 00000001:00080000:3.0:1789762725.487728:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967453 is committed 00000100:00100000:0.0:1789762725.501033:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067271296 00000100:00100000:0.0:1789762725.501044:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067271296 00000100:00100000:0.0:1789762725.501098:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 198903324351 time 198706281550 state 0x0 ref 1 00000100:00100000:0.0:1789762725.501108:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 198906281550 time 198706281550 state 0x1 ref 1 00010000:00010000:0.0:1789762725.504035:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762725.659314:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 198906281550 time 198906383689 state 0x0 ref 1 00000100:00100000:2.0:1789762725.659349:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6094380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067271296:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762725.659423:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 1048576, mtime: 0x6aad9ca5, atime: 0x6aad9ca5, ctime: 0x6aad9ca5, blocks: 0x803 00010000:00010000:2.0:1789762725.659432:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174632 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.659450:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=198s v=(0 0) 00010000:00010000:2.0:1789762725.659455:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762725.659462:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174632 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762725.659478:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174632 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762725.673459:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967454 is committed 00002000:00010000:3.0:1789762725.673486:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:3.0:1789762725.673491:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:3.0:1789762725.673505:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174632 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.673520:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174632 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.673531:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174632 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.673553:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cec00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762725.673559:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174632 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.673570:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174632 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.673589:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174632 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.673598:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174632 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762725.673679:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6094380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067271296:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14344us (172717us total) trans 4294967454 rc 0/0 00000100:00100000:3.0:1789762725.673688:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 36 00000100:00100000:3.0:1789762725.673695:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 198906281550 time 198906383689 state 0x2 ref 0 00000100:00100000:3.0:1789762725.676049:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762725.676112:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067271424 00000100:00100000:2.0:1789762725.676121:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067271424 00000100:00100000:2.0:1789762725.676165:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 198906281550 time 198906383689 state 0x0 ref 1 00000100:00100000:2.0:1789762725.676177:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199106383689 time 198906383689 state 0x1 ref 1 00000100:00100000:3.0:1789762725.859427:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199106383689 time 199106496762 state 0x0 ref 1 00000100:00100000:3.0:1789762725.859460:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067271424:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762725.859519:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 2097152, mtime: 0x6aad9ca5, atime: 0x6aad9ca5, ctime: 0x6aad9ca5, blocks: 0x1007 00010000:00010000:3.0:1789762725.859526:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174639 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.859556:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=199s v=(0 0) 00010000:00010000:3.0:1789762725.859559:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762725.859564:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174639 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.859578:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174639 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762725.871301:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967455 is committed 00002000:00010000:3.0:1789762725.871323:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:3.0:1789762725.871330:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:3.0:1789762725.871344:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174639 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.871357:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174639 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.871366:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174639 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.871374:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762725.871379:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174639 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.871389:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174639 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.871438:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174639 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762725.871447:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174639 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762725.871521:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067271424:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12072us (195469us total) trans 4294967455 rc 0/0 00000100:00100000:3.0:1789762725.871532:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 37 00000100:00100000:3.0:1789762725.871540:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199106383689 time 199106496762 state 0x2 ref 0 00000100:00100000:0.0:1789762725.873655:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:0.0:1789762726.072987:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967456 is committed 00002000:00010000:3.0:1789762726.073017:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:3.0:1789762726.073022:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:3.0:1789762726.073032:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f174640 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.073045:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f174640 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.073055:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f174640 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.073064:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762726.073069:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f174640 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.073090:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f174640 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.073105:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f174640 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.073111:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f174640 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762726.073177:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067271552:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13668us (199520us total) trans 4294967456 rc 0/0 00000100:00100000:3.0:1789762726.073185:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 38 00000100:00100000:3.0:1789762726.073190:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199306496762 time 199306562920 state 0x2 ref 0 00000100:00100000:3.0:1789762726.075693:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067271680 00000100:00100000:3.0:1789762726.075702:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067271680 00000100:00100000:3.0:1789762726.075733:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199306496762 time 199306562920 state 0x0 ref 1 00000100:00100000:3.0:1789762726.075741:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199506562920 time 199306562920 state 0x1 ref 1 00000100:00100000:0.0:1789762726.259581:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199506562920 time 199506653840 state 0x0 ref 1 00000100:00100000:0.0:1789762726.259609:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067271680:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762726.259660:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 4194304, mtime: 0x6aad9ca5, atime: 0x6aad9ca5, ctime: 0x6aad9ca5, blocks: 0x2007 00010000:00010000:0.0:1789762726.259680:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174647 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762726.259692:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=199s v=(0 0) 00010000:00010000:0.0:1789762726.259695:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762726.259700:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174647 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762726.259711:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174647 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762726.272347:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967457 is committed 00002000:00010000:3.0:1789762726.272363:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:3.0:1789762726.272368:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:3.0:1789762726.272381:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174647 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.272390:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174647 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.272422:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174647 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.272428:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762726.272431:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174647 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.272438:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174647 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.272453:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174647 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.272458:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174647 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762726.272527:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067271680:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12928us (196864us total) trans 4294967457 rc 0/0 00000100:00100000:3.0:1789762726.272537:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 39 00000100:00100000:3.0:1789762726.272544:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199506562920 time 199506653840 state 0x2 ref 0 00000100:00100000:3.0:1789762726.274807:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000004:00080000:2.0:1789762726.400093:0:9022:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 199647171282ns, next at 200647167449ns, rc = 0 00000004:00080000:3.0:1789762726.400093:0:9018:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 199647171434ns, next at 200647167697ns, rc = 0 00000100:00100000:2.0:1789762726.400135:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074614528:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762726.400151:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074614784:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.400169:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ff9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074614656:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762726.400236:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762726.400236:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074614912:0@lo:400:kworker.0 00000100:00100000:3.0:1789762726.400239:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762726.400282:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762726.400304:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074615168:0@lo:400:kworker.0 00000100:00100000:3.0:1789762726.400315:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074615296:0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.400316:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074615424:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.400347:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762726.400368:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762726.400370:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074614912 00000100:00100000:2.0:1789762726.400376:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074614912 00000100:00100000:0.0:1789762726.400380:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074615680:0@lo:400:kworker.0 00000100:00100000:3.0:1789762726.400414:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074615808:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.400422:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762726.400431:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074615168 00000100:00100000:2.0:1789762726.400435:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074615168 00000100:00100000:3.0:1789762726.400452:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762726.400458:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074616192:0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.400466:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074615680 00000100:00100000:2.0:1789762726.400471:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074615680 00000100:00100000:3.0:1789762726.400484:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074616320:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.400494:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762726.400494:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074615808 00000100:00100000:2.0:1789762726.400497:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074615808 00000100:00100000:3.0:1789762726.400522:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762726.400527:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:2.0:1789762726.400538:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074614912:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.400551:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ff9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074614656:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762726.400566:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074614784 00000100:00100000:3.0:1789762726.400570:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074614784 00000100:00100000:2.0:1789762726.400583:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074614912:12345-0@lo:400:kworker.0 Request processed in 46us (300us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762726.400592:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:0.0:1789762726.400594:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074616192 00000100:00100000:0.0:1789762726.400598:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074616192 00000100:00100000:3.0:1789762726.400616:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:2.0:1789762726.400621:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:3.0:1789762726.400629:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1551850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074614784:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.400638:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074616320 00000100:00100000:2.0:1789762726.400638:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074615680:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.400643:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074616320 00000100:00100000:3.0:1789762726.400676:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1551850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074614784:12345-0@lo:400:kworker.0 Request processed in 51us (444us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762726.400682:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 83 00000100:00100000:2.0:1789762726.400686:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074615680:12345-0@lo:400:kworker.0 Request processed in 55us (264us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762726.400690:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00100000:0.0:1789762726.400701:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074616192:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.400701:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00100000:3.0:1789762726.400735:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074615296 00000100:00100000:3.0:1789762726.400740:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074615296 00000100:00100000:0.0:1789762726.400751:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074616192:12345-0@lo:400:kworker.0 Request processed in 52us (256us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762726.400756:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 84 00000100:00100000:0.0:1789762726.400762:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 83 00000100:00100000:2.0:1789762726.400766:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074616320:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762726.400775:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 156 00000100:00100000:3.0:1789762726.400785:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074615296:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762726.400794:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762726.400802:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762726.400823:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074616320:12345-0@lo:400:kworker.0 Request processed in 58us (301us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762726.400831:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762726.400840:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 84 00000100:00100000:3.0:1789762726.400841:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074615296:12345-0@lo:400:kworker.0 Request processed in 53us (470us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762726.400853:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 156 00000100:00100000:0.0:1789762726.400861:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074615040:0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.400889:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762726.400897:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762726.400899:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074616320:0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.400936:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074616064:0@lo:400:kworker.0 00000100:00100000:3.0:1789762726.400940:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074614784:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.400952:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074615552:0@lo:400:kworker.0 00000100:00100000:3.0:1789762726.400961:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074615296:0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.400973:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762726.400978:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074615808:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.400985:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762726.400997:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074616448:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.401001:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d19f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074615936:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762726.401035:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762726.401035:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762726.401045:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074615040 00000100:00100000:3.0:1789762726.401049:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074615040 00000100:00100000:0.0:1789762726.401062:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074616576:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762726.401078:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074614912:0@lo:400:kworker.0 00000100:00100000:3.0:1789762726.401083:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:0.0:1789762726.401091:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762726.401097:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074615040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.401112:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074616064 00000100:00100000:2.0:1789762726.401118:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074616064 00000100:00100000:0.0:1789762726.401130:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074614528:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762726.401150:0:3425:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967340 gen 2 00000100:00100000:2.0:1789762726.401154:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074616448 00000100:00100000:3.0:1789762726.401155:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074615040:12345-0@lo:400:kworker.0 Request processed in 59us (257us total) trans 0 rc 0/0 00000004:00080000:0.0:1789762726.401158:0:3425:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967340 committed transno 4294967340 00000100:00100000:2.0:1789762726.401158:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074616448 00000100:00100000:3.0:1789762726.401167:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:0.0:1789762726.401184:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074615040:0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.401188:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 85 00000100:00100000:2.0:1789762726.401198:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074616064:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762726.401206:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 93 00000100:00100000:3.0:1789762726.401218:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074616576:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762726.401224:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d19f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074615936:0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789762726.401234:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 0 free 3859808256 avail 3835691008 00000100:00100000:2.0:1789762726.401248:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074616064:12345-0@lo:400:kworker.0 Request processed in 51us (278us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762726.401257:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 85 00000100:00080000:0.0:1789762726.401261:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762726.401276:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 86 00000100:00100000:3.0:1789762726.401282:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074616576:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (189us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762726.401287:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074616448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.401290:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074616576:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762726.401297:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 93 00000100:00100000:3.0:1789762726.401325:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074615552 00000100:00100000:3.0:1789762726.401330:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074615552 00000100:00100000:2.0:1789762726.401332:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074616448:12345-0@lo:400:kworker.0 Request processed in 46us (294us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762726.401341:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762726.401342:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 86 00000100:00080000:0.0:1789762726.401345:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762726.401376:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:2.0:1789762726.401388:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074615552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762726.401445:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074615424:0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.401455:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074615552:12345-0@lo:400:kworker.0 Request processed in 69us (472us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762726.401465:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:3.0:1789762726.401465:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074616064:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.401478:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074615552:0@lo:400:kworker.0 00000100:00100000:3.0:1789762726.401490:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074616448:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.459658:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199706653840 time 199706729575 state 0x0 ref 1 00000100:00100000:0.0:1789762726.459684:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067271808:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762726.459742:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 5242880, mtime: 0x6aad9ca5, atime: 0x6aad9ca5, ctime: 0x6aad9ca5, blocks: 0x2807 00010000:00010000:0.0:1789762726.459751:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17464e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762726.459764:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=199s v=(0 0) 00010000:00010000:0.0:1789762726.459769:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762726.459773:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17464e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762726.459783:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17464e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762726.471934:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967458 is committed 00002000:00010000:0.0:1789762726.471941:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:0.0:1789762726.471947:0:8058:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb mtime from disk: 1789762725 -> 1789762726 00002000:00010000:0.0:1789762726.471950:0:8058:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb ctime from disk: 1789762725 -> 1789762726 00002000:00010000:0.0:1789762726.471952:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:0.0:1789762726.471963:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17464e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762726.471976:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17464e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762726.471986:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17464e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762726.472002:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762726.472007:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17464e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762726.472017:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17464e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762726.472037:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17464e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762726.472046:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17464e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762726.472119:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067271808:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12443us (197308us total) trans 4294967458 rc 0/0 00000100:00100000:0.0:1789762726.472130:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 40 00000100:00100000:0.0:1789762726.472138:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199706653840 time 199706729575 state 0x2 ref 0 00000100:00100000:3.0:1789762726.474614:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762726.474701:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067271936 00000100:00100000:2.0:1789762726.474709:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067271936 00000100:00100000:2.0:1789762726.474742:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199706653840 time 199706729575 state 0x0 ref 1 00000100:00100000:2.0:1789762726.474751:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199906729575 time 199706729575 state 0x1 ref 1 00010000:00010000:0.0:1789762726.528088:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762726.659708:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199906729575 time 199906780847 state 0x0 ref 1 00000100:00100000:2.0:1789762726.659737:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067271936:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762726.659795:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 6291456, mtime: 0x6aad9ca6, atime: 0x6aad9ca5, ctime: 0x6aad9ca6, blocks: 0x3007 00010000:00010000:2.0:1789762726.659803:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000235778e/0xdf5a7ac59f174655 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762726.659816:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=199s v=(0 0) 00010000:00010000:2.0:1789762726.659820:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762726.659825:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000235778e/0xdf5a7ac59f174655 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762726.659837:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000235778e/0xdf5a7ac59f174655 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762726.670557:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:3.0:1789762726.670563:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 12295 -> 14343 00010000:00010000:3.0:1789762726.670575:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000235778e/0xdf5a7ac59f174655 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.670588:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000235778e/0xdf5a7ac59f174655 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.670606:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000235778e/0xdf5a7ac59f174655 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.670614:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cfe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762726.670619:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000235778e/0xdf5a7ac59f174655 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.670626:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000235778e/0xdf5a7ac59f174655 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.670641:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000235778e/0xdf5a7ac59f174655 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.670646:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000235778e/0xdf5a7ac59f174655 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762726.670705:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067271936:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10981us (196091us total) trans 4294967459 rc 0/0 00000100:00100000:3.0:1789762726.670714:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 41 00000100:00100000:3.0:1789762726.670720:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199906729575 time 199906780847 state 0x2 ref 0 00000100:00100000:3.0:1789762726.673010:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067272064 00000100:00100000:3.0:1789762726.673019:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067272064 00000100:00100000:3.0:1789762726.673057:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 199906729575 time 199906780847 state 0x0 ref 1 00000100:00100000:3.0:1789762726.673066:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 200106780847 time 199906780847 state 0x1 ref 1 00000001:00080000:2.0:1789762726.871542:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967460 is committed 00002000:00010000:3.0:1789762726.871570:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:3.0:1789762726.871575:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:3.0:1789762726.871584:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f17465c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.871593:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f17465c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.871599:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f17465c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.871604:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762726.871607:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f17465c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.871615:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f17465c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.871628:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f17465c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762726.871633:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f17465c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762726.871687:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067272064:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11889us (198714us total) trans 4294967460 rc 0/0 00000100:00100000:3.0:1789762726.871695:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 42 00000100:00100000:3.0:1789762726.871707:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 200106780847 time 200106854014 state 0x2 ref 0 00000100:00100000:3.0:1789762726.873715:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762726.873762:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067272192 00000100:00100000:0.0:1789762726.873769:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067272192 00000100:00100000:0.0:1789762726.873801:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 200106780847 time 200106854014 state 0x0 ref 1 00000100:00100000:0.0:1789762726.873810:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 200306854014 time 200106854014 state 0x1 ref 1 00000001:00080000:3.0:1789762726.933186:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967337 is committed 00000001:00080000:3.0:1789762726.933205:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967343 is committed 00000100:00100000:3.0:1789762727.059874:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 200306854014 time 200306940510 state 0x0 ref 1 00000100:00080000:0.0:1789762727.059942:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762727.059944:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067272192:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762727.060005:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 8388608, mtime: 0x6aad9ca6, atime: 0x6aad9ca5, ctime: 0x6aad9ca6, blocks: 0x4007 00010000:00010000:3.0:1789762727.060020:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174663 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762727.060033:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=200s v=(0 0) 00010000:00010000:3.0:1789762727.060037:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762727.060042:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174663 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762727.060054:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f174663 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762727.071668:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967461 is committed 00000100:00100000:2.0:1789762727.073989:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762727.074028:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067272320 00000100:00100000:3.0:1789762727.074037:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067272320 00000100:00100000:3.0:1789762727.074079:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 200306854014 time 200306940510 state 0x0 ref 1 00000100:00100000:3.0:1789762727.074090:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 200506940510 time 200306940510 state 0x1 ref 1 00000001:00080000:0.0:1789762727.276228:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967462 is committed 00000100:00100000:2.0:1789762727.279254:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762727.279309:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067272448 00000100:00100000:3.0:1789762727.279319:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067272448 00000100:00100000:3.0:1789762727.279369:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 200506940510 time 200507032842 state 0x0 ref 1 00000100:00100000:3.0:1789762727.279380:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 200707032842 time 200507032842 state 0x1 ref 1 00010000:00010000:0.0:1789762727.435193:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174480 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435213:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174480 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435223:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174480 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435253:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cea00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762727.435264:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174480 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435289:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174480 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435320:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x73550ecb (ffff9c37d3bee000) 00010000:00010000:0.0:1789762727.435326:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762727.435334:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174480 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435346:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee8cea00/0xdf5a7ac59f174480 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x6:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762727.435367:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ff9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074616832:0@lo:6:osp-syn-0-1.0 00000020:00080000:2.0:1789762727.435374:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 4 committed batchid 3 00010000:00010000:0.0:1789762727.435382:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934000/0xdf5a7ac59f17449c lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435427:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934000/0xdf5a7ac59f17449c lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435432:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934000/0xdf5a7ac59f17449c lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435446:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762727.435450:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934000/0xdf5a7ac59f17449c lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435456:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934000/0xdf5a7ac59f17449c lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435466:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x6:0x0].0x2b0e463d (ffff9c37c72bae00) 00010000:00010000:0.0:1789762727.435469:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762727.435473:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934000/0xdf5a7ac59f17449c lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435497:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934000/0xdf5a7ac59f17449c lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x6:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762727.435545:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074616960 00000100:00100000:0.0:1789762727.435549:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074617088:0@lo:6:osp-syn-1-1.0 00000100:00100000:3.0:1789762727.435555:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074616960 00000100:00100000:0.0:1789762727.435616:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762727.435630:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074617088 00000100:00100000:3.0:1789762727.435633:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074617088 00000100:00100000:3.0:1789762727.435671:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 96 00000100:00100000:3.0:1789762727.435682:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074617088:12345-0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1789762727.435689:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00080000:3.0:1789762727.435701:0:9038:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:5 count 1 00000100:00100000:0.0:1789762727.435714:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789762727.435721:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074616704:0@lo:6:osp-syn-1-1.0 00002000:00010000:3.0:1789762727.435726:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x5:0x0] initial LVB size: 8388608, mtime: 0x6aad9ca3, atime: 0x6aad9ca1, ctime: 0x6aad9ca3, blocks: 0x4007 00010000:00010000:3.0:1789762727.435734:0:9038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f174678 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762727.435757:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1789762727.435761:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x5:0x0].0x0 since=200s v=(0 0) 00010000:00010000:3.0:1789762727.435771:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x5:0x0].0x0 since=0s v=(0 0) 00000004:00080000:2.0:1789762727.435780:0:7233:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37c9fc8600 with version 10 00000100:00100000:0.0:1789762727.435783:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074616832 00010000:00010000:3.0:1789762727.435785:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f174678 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762727.435789:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074616832 00000020:00080000:2.0:1789762727.435798:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 3 cancel update log [0x200000401:0x2:0x0].3: rc = 0 00000100:00100000:0.0:1789762727.435815:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074616704 00010000:00010000:3.0:1789762727.435818:0:9038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f174678 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762727.435819:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074616704 00000100:00100000:0.0:1789762727.435837:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 97 00010000:00010000:3.0:1789762727.435843:0:9038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f174678 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762727.435850:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074616832:12345-0@lo:6:osp-syn-0-1.0 00010000:00010000:3.0:1789762727.435885:0:9038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f174678 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1789762727.435895:0:9037:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:4 count 1 00010000:00010000:3.0:1789762727.435916:0:9038:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1789762727.435921:0:9037:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17467f lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435967:0:9037:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f174441 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4a4 expref: 6 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.435975:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=2s v=(0 0) 00010000:00010000:0.0:1789762727.435995:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762727.435999:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17467f lrc: 4/0,1 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.436023:0:9037:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f174441 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4a4 expref: 6 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.436036:0:9037:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f174441 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4a4 expref: 6 pid: 8053 timeout: 300 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762727.436045:0:9037:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d98af800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9037:1876702074617216:192.168.203.37@tcp:104: 00000100:00100000:0.0:1789762727.436110:0:9037:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d2590d80 going to sleep for 11 seconds 00000004:00080000:0.0:1789762727.436121:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 10 rpc_version 10 00000020:00100000:0.0:1789762727.436139:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762727.436146:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37d98adf80 x1876702074617472/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762727.436158:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d98adf80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074617472:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762727.436193:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762727.436209:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d2590540 going to sleep for 11 seconds 00010000:00010000:0.0:1789762727.436221:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1789762727.436232:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f174671 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.436239:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762727.436242:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f174671 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.436249:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f174671 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.436261:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f174671 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.436266:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f174671 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.436299:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762727.436303:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f174678 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.436310:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762727.436313:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f174678 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.436320:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f174678 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.436342:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f174678 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.436349:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f174678 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.436364:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762727.436366:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762727.436370:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762727.436402:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074617472 00000100:00100000:0.0:1789762727.436407:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074617472 00000100:00100000:0.0:1789762727.436462:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 158 00000100:00100000:0.0:1789762727.436469:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9a37850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:7224:x1876702074617472:12345-0@lo:1000:osp_up0-1.0 00000020:00080000:2.0:1789762727.436784:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 3 cancel update log [0x240000400:0x2:0x0].3: rc = 0 00000020:00080000:2.0:1789762727.436815:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 3 00010000:00010000:2.0:1789762727.436851:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762727.436981:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9a37850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074617472:12345-0@lo:1000:osp_up0-1.0 Request processed in 509us (785us total) trans 4294967353 rc 0/0 00000100:00100000:0.0:1789762727.436991:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 158 00000100:00100000:2.0:1789762727.437028:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d98adf80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074617472:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1789762727.437240:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+7:3425:x1876702074617088:12345-0@lo:6:osp-syn-1-1.0 Request processed in 1556us (1620us total) trans 4294967344 rc 0/0 00000100:00100000:3.0:1789762727.437253:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 96 00000004:00080000:2.0:1789762727.437296:0:3425:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c36c9ffc380/2, rc 0, transno 4294967344 00000100:00100000:3.0:1789762727.437297:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 98 00000100:00100000:2.0:1789762727.437312:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074617088:0@lo:6:osp-syn-1-1.0 00000100:00100000:3.0:1789762727.437323:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074616704:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:3.0:1789762727.437346:0:9038:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:4 count 1 00002000:00010000:3.0:1789762727.437372:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x4:0x0] initial LVB size: 8388608, mtime: 0x6aad9ca2, atime: 0x6aad9ca1, ctime: 0x6aad9ca2, blocks: 0x4007 00010000:00010000:3.0:1789762727.437380:0:9038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000032f2d923/0xdf5a7ac59f174686 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762727.437406:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4:0x0].0x0 since=200s v=(0 0) 00010000:00010000:3.0:1789762727.437411:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762727.437416:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000032f2d923/0xdf5a7ac59f174686 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762727.437427:0:9038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000032f2d923/0xdf5a7ac59f174686 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762727.437451:0:9038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000032f2d923/0xdf5a7ac59f174686 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762727.437456:0:9038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000032f2d923/0xdf5a7ac59f174686 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762727.437477:0:9038:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762727.437485:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762727.437491:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000032f2d923/0xdf5a7ac59f174686 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.437515:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762727.437519:0:6408:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000032f2d923/0xdf5a7ac59f174686 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.437528:0:6408:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000032f2d923/0xdf5a7ac59f174686 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.437549:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000032f2d923/0xdf5a7ac59f174686 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.437555:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000032f2d923/0xdf5a7ac59f174686 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.437568:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762727.437570:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762727.437571:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762727.437876:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+7:3427:x1876702074616704:12345-0@lo:6:osp-syn-1-1.0 Request processed in 565us (2122us total) trans 4294967345 rc 0/0 00000100:00100000:3.0:1789762727.437882:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 98 00000004:00080000:0.0:1789762727.437899:0:3427:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c36c9ffc700/2, rc 0, transno 4294967345 00000100:00100000:0.0:1789762727.437905:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074616704:0@lo:6:osp-syn-1-1.0 00000100:00100000:3.0:1789762727.439099:0:9037:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d98af800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9037:1876702074617216:192.168.203.37@tcp:104: 00010000:00010000:3.0:1789762727.439117:0:9037:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17467f lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762727.439162:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:3.0:1789762727.439367:0:9037:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17467f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762727.439384:0:9037:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17467f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762727.439410:0:9037:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17467f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762727.439419:0:9037:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17467f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762727.439460:0:9037:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1789762727.440157:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+7:3427:x1876702074616832:12345-0@lo:6:osp-syn-0-1.0 Request processed in 4309us (4468us total) trans 4294967464 rc 0/0 00000100:00100000:3.0:1789762727.440167:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 97 00000004:00080000:0.0:1789762727.440171:0:3427:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c36c9ff9f80/2, rc 0, transno 4294967464 00000100:00100000:0.0:1789762727.440180:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ff9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074616832:0@lo:6:osp-syn-0-1.0 00000100:00100000:2.0:1789762727.460086:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 200707032842 time 200707157889 state 0x0 ref 1 00000100:00100000:2.0:1789762727.460115:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067272448:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762727.460181:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 10485760, mtime: 0x6aad9ca6, atime: 0x6aad9ca5, ctime: 0x6aad9ca6, blocks: 0x5007 00010000:00010000:2.0:1789762727.460188:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f17468d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762727.460201:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=200s v=(0 0) 00010000:00010000:2.0:1789762727.460204:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762727.460208:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f17468d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762727.460236:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f17468d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762727.477951:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967463 is committed 00000001:00080000:0.0:1789762727.477969:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967465 is committed 00000001:00080000:0.0:1789762727.477996:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967464 is committed 00002000:00010000:2.0:1789762727.478002:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:2.0:1789762727.478007:0:13300:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb mtime from disk: 1789762726 -> 1789762727 00002000:00010000:2.0:1789762727.478011:0:13300:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb ctime from disk: 1789762726 -> 1789762727 00002000:00010000:2.0:1789762727.478014:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:2.0:1789762727.478028:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f17468d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762727.478052:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f17468d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762727.478064:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f17468d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762727.478073:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8ce200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762727.478079:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f17468d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762727.478090:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f17468d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762727.478109:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f17468d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762727.478117:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ccca588f/0xdf5a7ac59f17468d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762727.478220:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067272448:12345-192.168.203.37@tcp:4:dd.500 Request processed in 18113us (198963us total) trans 4294967465 rc 0/0 00000100:00100000:2.0:1789762727.478232:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 45 00000100:00100000:2.0:1789762727.478240:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 200707032842 time 200707157889 state 0x2 ref 0 00000100:00100000:2.0:1789762727.480562:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:0.0:1789762727.552051:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789762727.673424:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967466 is committed 00000100:00100000:0.0:1789762727.675803:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762727.860274:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 201107213469 time 201107345736 state 0x0 ref 1 00000100:00100000:0.0:1789762727.860315:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067272832:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762727.860370:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 12582912, mtime: 0x6aad9ca7, atime: 0x6aad9ca5, ctime: 0x6aad9ca7, blocks: 0x6007 00010000:00010000:0.0:1789762727.860379:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f17469b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.860448:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=201s v=(0 0) 00010000:00010000:0.0:1789762727.860452:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762727.860458:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f17469b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762727.860491:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f17469b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762727.875404:0:14225:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967467 is committed 00000100:00100000:0.0:1789762727.877792:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762727.877849:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067272960 00000100:00100000:2.0:1789762727.877859:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067272960 00000100:00100000:2.0:1789762727.877920:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 201107213469 time 201107345736 state 0x0 ref 1 00000100:00100000:2.0:1789762727.877931:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 201307345736 time 201107345736 state 0x1 ref 1 00000100:00100000:3.0:1789762728.060424:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 201307345736 time 201307491659 state 0x0 ref 1 00000100:00100000:3.0:1789762728.060465:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067272960:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762728.060535:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 13631488, mtime: 0x6aad9ca7, atime: 0x6aad9ca5, ctime: 0x6aad9ca7, blocks: 0x6807 00010000:00010000:3.0:1789762728.060548:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.060564:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=201s v=(0 0) 00010000:00010000:3.0:1789762728.060568:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762728.060574:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.060588:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762728.076231:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 13631488 -> 14680064 00002000:00010000:3.0:1789762728.076237:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:3.0:1789762728.076249:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.076261:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.076269:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.076310:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762728.076315:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.076323:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.076341:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.076348:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762728.076476:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067272960:12345-192.168.203.37@tcp:4:dd.500 Request processed in 16024us (198679us total) trans 4294967468 rc 0/0 00000100:00100000:3.0:1789762728.076492:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 48 00000100:00100000:3.0:1789762728.076502:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 201307345736 time 201307491659 state 0x2 ref 0 00000100:00100000:2.0:1789762728.078711:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762728.260555:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 201507491659 time 201507626237 state 0x0 ref 1 00000100:00100000:3.0:1789762728.260585:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067273088:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762728.260640:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 14680064, mtime: 0x6aad9ca7, atime: 0x6aad9ca5, ctime: 0x6aad9ca7, blocks: 0x7007 00010000:00010000:3.0:1789762728.260648:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.260698:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=201s v=(0 0) 00010000:00010000:3.0:1789762728.260702:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762728.260707:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.260719:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762728.274313:0:14225:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967469 is committed 00002000:00010000:3.0:1789762728.274375:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 14680064 -> 15728640 00002000:00010000:3.0:1789762728.274397:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:3.0:1789762728.274409:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.274420:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.274427:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.274433:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762728.274437:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.274445:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.274474:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.274481:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762728.274549:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067273088:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13977us (195839us total) trans 4294967469 rc 0/0 00000100:00100000:3.0:1789762728.274558:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 49 00000100:00100000:3.0:1789762728.274564:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 201507491659 time 201507626237 state 0x2 ref 0 00000100:00100000:0.0:1789762728.276868:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762728.448125:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074617600:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762728.448212:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762728.448511:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074617600:0@lo:41:osp-pre-1-0.0 00000001:00080000:0.0:1789762728.474310:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967470 is committed 00000100:00100000:2.0:1789762728.477357:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762728.477438:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067273344 00000100:00100000:3.0:1789762728.477446:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067273344 00000100:00100000:3.0:1789762728.477482:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 201707626237 time 201707687509 state 0x0 ref 1 00000100:00100000:3.0:1789762728.477492:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 201907687509 time 201707687509 state 0x1 ref 1 00010000:00010000:2.0:1789762728.576075:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762728.660729:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 201907687509 time 201907797225 state 0x0 ref 1 00000100:00100000:3.0:1789762728.660788:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1773b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067273344:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762728.660869:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 16777216, mtime: 0x6aad9ca8, atime: 0x6aad9ca5, ctime: 0x6aad9ca8, blocks: 0x8007 00010000:00010000:3.0:1789762728.660878:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.660893:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=201s v=(0 0) 00010000:00010000:3.0:1789762728.660897:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762728.660918:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.660932:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762728.681795:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 16777216 -> 17825792 00002000:00010000:3.0:1789762728.681802:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:3.0:1789762728.681813:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.681825:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.681833:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.681840:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762728.681843:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.681852:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.681868:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762728.681887:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762728.682006:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1773b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067273344:12345-192.168.203.37@tcp:4:dd.500 Request processed in 21236us (204647us total) trans 4294967471 rc 0/0 00000100:00100000:3.0:1789762728.682016:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 51 00000100:00100000:3.0:1789762728.682024:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 201907687509 time 201907797225 state 0x2 ref 0 00000100:00100000:0.0:1789762728.685227:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762728.685283:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067273472 00000100:00100000:3.0:1789762728.685296:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067273472 00000100:00100000:3.0:1789762728.685344:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 201907687509 time 201907797225 state 0x0 ref 1 00000100:00100000:3.0:1789762728.685359:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 202107797225 time 201907797225 state 0x1 ref 1 00000100:00100000:2.0:1789762728.860810:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 202107797225 time 202107879462 state 0x0 ref 1 00000100:00100000:2.0:1789762728.860841:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067273472:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762728.860889:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 17825792, mtime: 0x6aad9ca8, atime: 0x6aad9ca5, ctime: 0x6aad9ca8, blocks: 0x8807 00010000:00010000:2.0:1789762728.860897:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f1746be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.860910:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=202s v=(0 0) 00010000:00010000:2.0:1789762728.860914:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762728.860940:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f1746be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.860953:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f1746be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762728.873936:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 17825792 -> 18874368 00000001:00080000:0.0:1789762728.873943:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967472 is committed 00002000:00010000:2.0:1789762728.873958:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 34823 -> 36871 00010000:00010000:2.0:1789762728.873974:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f1746be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.873985:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f1746be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.873992:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f1746be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.873997:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762728.874001:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f1746be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.874008:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f1746be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.874043:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f1746be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.874052:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f1746be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762728.874144:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067273472:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13297us (188906us total) trans 4294967472 rc 0/0 00000100:00100000:2.0:1789762728.874159:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 52 00000100:00100000:2.0:1789762728.874166:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 202107797225 time 202107879462 state 0x2 ref 0 00000100:00100000:3.0:1789762728.876844:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00080000:0.0:1789762729.061011:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789762729.079496:0:14225:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967473 is committed 00002000:00010000:3.0:1789762729.079528:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:3.0:1789762729.079535:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:3.0:1789762729.079551:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f1746c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762729.079563:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f1746c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762729.079571:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f1746c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762729.079577:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762729.079590:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f1746c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762729.079599:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f1746c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762729.079621:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f1746c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762729.079628:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f1746c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762729.079748:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067273600:12345-192.168.203.37@tcp:4:dd.500 Request processed in 18836us (202899us total) trans 4294967473 rc 0/0 00000100:00100000:3.0:1789762729.079762:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 53 00000100:00100000:3.0:1789762729.079771:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 202307879462 time 202307958839 state 0x2 ref 0 00000100:00100000:0.0:1789762729.082671:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762729.082741:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067273728 00000100:00100000:3.0:1789762729.082752:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067273728 00000100:00100000:3.0:1789762729.082803:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 202307879462 time 202307958839 state 0x0 ref 1 00000100:00100000:3.0:1789762729.082816:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 202507958839 time 202307958839 state 0x1 ref 1 00000001:00080000:0.0:1789762729.280813:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967474 is committed 00000100:00100000:0.0:1789762729.284026:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762729.284097:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067273856 00000100:00100000:2.0:1789762729.284110:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067273856 00000100:00100000:2.0:1789762729.284162:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 202507958839 time 202508103730 state 0x0 ref 1 00000100:00100000:2.0:1789762729.284190:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 202708103730 time 202508103730 state 0x1 ref 1 00000100:00100000:3.0:1789762729.461148:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 202708103730 time 202708214745 state 0x0 ref 1 00000100:00100000:3.0:1789762729.461188:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067273856:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762729.461275:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 20971520, mtime: 0x6aad9ca8, atime: 0x6aad9ca5, ctime: 0x6aad9ca8, blocks: 0xa007 00010000:00010000:3.0:1789762729.461284:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762729.461331:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=202s v=(0 0) 00010000:00010000:3.0:1789762729.461337:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762729.461346:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762729.461361:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f1746d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762729.472173:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762729.472199:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074617728:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762729.472274:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762729.472392:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074617728 00000100:00100000:0.0:1789762729.472403:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074617728 00000100:00100000:0.0:1789762729.472493:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:0.0:1789762729.472515:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3426:x1876702074617728:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762729.472582:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762729.472686:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3426:x1876702074617728:12345-0@lo:41:osp-pre-0-1.0 Request processed in 177us (410us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762729.472699:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:0.0:1789762729.472798:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074617728:0@lo:41:osp-pre-0-1.0 00000001:00080000:3.0:1789762729.476267:0:14225:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967475 is committed 00000100:00100000:3.0:1789762729.479384:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:0.0:1789762729.600124:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762729.676086:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967476 is committed 00000100:00100000:0.0:1789762729.678580:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:3.0:1789762729.873850:0:14225:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967477 is committed 00000100:00100000:0.0:1789762729.876481:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:3.0:1789762730.073714:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967478 is committed 00000100:00100000:3.0:1789762730.076278:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762730.261466:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 203508396382 time 203508534137 state 0x0 ref 1 00000100:00100000:2.0:1789762730.261514:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067274368:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762730.261586:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 25165824, mtime: 0x6aad9ca9, atime: 0x6aad9ca5, ctime: 0x6aad9ca9, blocks: 0xc007 00010000:00010000:2.0:1789762730.261597:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1746ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.261620:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=203s v=(0 0) 00010000:00010000:2.0:1789762730.261626:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762730.261634:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1746ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.261654:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1746ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762730.273943:0:14225:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967479 is committed 00002000:00010000:2.0:1789762730.273952:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:2.0:1789762730.273958:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 49159 -> 51209 00010000:00010000:2.0:1789762730.273972:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1746ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.273985:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1746ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.273992:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1746ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.273997:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cf000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762730.274001:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1746ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.274012:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1746ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.274038:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1746ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.274048:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1746ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762730.274149:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067274368:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12658us (197865us total) trans 4294967479 rc 0/0 00000100:00100000:2.0:1789762730.274169:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 59 00000100:00100000:2.0:1789762730.274177:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 203508396382 time 203508534137 state 0x2 ref 0 00000100:00100000:0.0:1789762730.276597:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762730.440167:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067274624 00000100:00100000:2.0:1789762730.440182:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067274624 00000100:00100000:0.0:1789762730.440212:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067274752 00000100:00100000:0.0:1789762730.440220:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067274752 00000100:00100000:3.0:1789762730.440252:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067274880 00000100:00100000:3.0:1789762730.440258:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067274880 00000100:00100000:2.0:1789762730.440270:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 305 00000100:00100000:2.0:1789762730.440291:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1550050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067274624:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762730.440299:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067275136 00000100:00100000:0.0:1789762730.440304:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067275136 00000100:00100000:3.0:1789762730.440309:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 219 00000100:00100000:3.0:1789762730.440320:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2397:x1876702067274752:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762730.440338:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067275264 00000100:00100000:0.0:1789762730.440343:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067275264 00000100:00100000:0.0:1789762730.440384:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 220 00000100:00100000:0.0:1789762730.440397:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067274880:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762730.440399:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2397:x1876702067274752:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 81us (262us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762730.440404:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1550050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067274624:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 118us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762730.440413:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 219 00000100:00100000:2.0:1789762730.440419:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 305 00000100:00100000:0.0:1789762730.440438:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067274880:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 42us (244us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762730.440449:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 221 00000100:00100000:0.0:1789762730.440455:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 220 00000100:00100000:3.0:1789762730.440459:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067275136:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762730.440475:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 222 00000100:00100000:0.0:1789762730.440487:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067275264:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762730.440487:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067275136:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 30us (210us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762730.440495:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 221 00000100:00100000:0.0:1789762730.440540:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067275264:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 53us (208us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762730.440550:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 222 00000100:00080000:3.0:1789762730.440553:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789762730.440559:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762730.440562:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762730.461573:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 203708534137 time 203708640375 state 0x0 ref 1 00000100:00100000:3.0:1789762730.461617:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067274496:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762730.461691:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 26214400, mtime: 0x6aad9ca9, atime: 0x6aad9ca5, ctime: 0x6aad9ca9, blocks: 0xc809 00010000:00010000:3.0:1789762730.461700:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008f5625c2/0xdf5a7ac59f1746f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762730.461718:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=203s v=(0 0) 00010000:00010000:3.0:1789762730.461722:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762730.461728:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008f5625c2/0xdf5a7ac59f1746f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762730.461742:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008f5625c2/0xdf5a7ac59f1746f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762730.477103:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967480 is committed 00000100:00100000:0.0:1789762730.480176:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:3.0:1789762730.504818:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967353 is committed 00000001:00080000:3.0:1789762730.504846:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967350 is committed 00010000:00010000:0.0:1789762730.504868:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762730.504875:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967352 is committed 00010000:00010000:0.0:1789762730.504884:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.504904:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.504940:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8ce400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762730.504950:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.504977:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505014:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff9c37d3bee600) 00010000:00010000:0.0:1789762730.505020:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762730.505026:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505045:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce400/0xdf5a7ac59f1744f7 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505068:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f1744fe lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505077:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f1744fe lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505084:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f1744fe lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505090:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762730.505093:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f1744fe lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505112:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f1744fe lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505123:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:0.0:1789762730.505125:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762730.505128:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f1744fe lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505134:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f1744fe lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505147:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174559 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505151:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174559 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505156:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174559 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505159:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762730.505161:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174559 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505164:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174559 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505169:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:0.0:1789762730.505170:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789762730.505172:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174559 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505183:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174559 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505192:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f17457c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505196:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f17457c lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505200:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f17457c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505204:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cfa00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762730.505206:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f17457c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505209:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f17457c lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505214:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:0.0:1789762730.505216:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762730.505217:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f17457c lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505221:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8cfa00/0xdf5a7ac59f17457c lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505227:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1745ad lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505231:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1745ad lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505239:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1745ad lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505243:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8ce000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762730.505245:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1745ad lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505249:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1745ad lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505254:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x4009f790 (ffff9c37d3bee500) 00010000:00010000:0.0:1789762730.505255:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762730.505257:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1745ad lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505260:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee8ce000/0xdf5a7ac59f1745ad lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505270:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f1745c2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505273:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f1745c2 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505277:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f1745c2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505280:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762730.505281:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f1745c2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505291:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f1745c2 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505297:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37db743c00) 00010000:00010000:0.0:1789762730.505298:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762730.505300:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f1745c2 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505304:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f1745c2 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505310:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f1745f3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505314:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f1745f3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505317:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f1745f3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505320:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762730.505322:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f1745f3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505326:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f1745f3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505331:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37db743c00) 00010000:00010000:0.0:1789762730.505333:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762730.505335:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f1745f3 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.505344:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f1745f3 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762730.661691:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 203908640375 time 203908755287 state 0x0 ref 1 00000100:00100000:0.0:1789762730.661741:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067275392:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762730.661824:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 27262976, mtime: 0x6aad9caa, atime: 0x6aad9ca5, ctime: 0x6aad9caa, blocks: 0xd009 00010000:00010000:0.0:1789762730.661836:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f1746fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.661858:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=203s v=(0 0) 00010000:00010000:0.0:1789762730.661863:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762730.661873:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f1746fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.661891:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f1746fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762730.675143:0:14225:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967481 is committed 00000100:00100000:2.0:1789762730.678105:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067275520 00000100:00100000:2.0:1789762730.678117:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067275520 00000100:00100000:2.0:1789762730.678175:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 203908640375 time 203908755287 state 0x0 ref 1 00000100:00100000:2.0:1789762730.678191:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 204108755287 time 203908755287 state 0x1 ref 1 00000100:00100000:2.0:1789762730.861769:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 204108755287 time 204108836134 state 0x0 ref 1 00000100:00100000:2.0:1789762730.861810:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067275520:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762730.861890:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 28311552, mtime: 0x6aad9caa, atime: 0x6aad9ca5, ctime: 0x6aad9caa, blocks: 0xd809 00010000:00010000:2.0:1789762730.861899:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174704 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.861933:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=204s v=(0 0) 00010000:00010000:2.0:1789762730.861937:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762730.861944:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174704 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.861979:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174704 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762730.874712:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967482 is committed 00000100:00100000:0.0:1789762730.877377:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762731.061872:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 204308836134 time 204308937980 state 0x0 ref 1 00000100:00100000:3.0:1789762731.061960:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067275648:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:0.0:1789762731.061995:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789762731.062032:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 29360128, mtime: 0x6aad9caa, atime: 0x6aad9ca5, ctime: 0x6aad9caa, blocks: 0xe009 00010000:00010000:3.0:1789762731.062059:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f17470b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762731.062077:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=204s v=(0 0) 00010000:00010000:3.0:1789762731.062081:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762731.062087:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f17470b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762731.062102:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f17470b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762731.074031:0:14225:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967483 is committed 00000100:00100000:2.0:1789762731.076611:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067275776 00000100:00100000:2.0:1789762731.076622:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067275776 00000100:00100000:2.0:1789762731.076666:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 204308836134 time 204308937980 state 0x0 ref 1 00000100:00100000:2.0:1789762731.076676:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 204508937980 time 204308937980 state 0x1 ref 1 00000100:00100000:2.0:1789762731.261981:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 204508937980 time 204509047975 state 0x0 ref 1 00000100:00100000:2.0:1789762731.262019:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067275776:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762731.262090:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 30408704, mtime: 0x6aad9caa, atime: 0x6aad9ca5, ctime: 0x6aad9caa, blocks: 0xe809 00010000:00010000:2.0:1789762731.262099:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174712 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762731.262117:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=204s v=(0 0) 00010000:00010000:2.0:1789762731.262121:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762731.262126:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174712 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762731.262140:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f174712 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762731.284534:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967484 is committed 00000100:00100000:0.0:1789762731.287836:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762731.462119:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 204709047975 time 204709185780 state 0x0 ref 1 00000100:00100000:3.0:1789762731.462159:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067275904:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762731.462233:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 31457280, mtime: 0x6aad9caa, atime: 0x6aad9ca5, ctime: 0x6aad9caa, blocks: 0xf009 00010000:00010000:3.0:1789762731.462242:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174719 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762731.462259:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=204s v=(0 0) 00010000:00010000:3.0:1789762731.462263:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762731.462269:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174719 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762731.462282:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174719 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762731.475706:0:14225:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967485 is committed 00000100:00100000:0.0:1789762731.478262:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762731.520099:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee760000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074618368:0@lo:400:kworker.0 00000100:00100000:3.0:1789762731.520188:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762731.520209:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d98adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074618240:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762731.520270:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b95f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074618880:0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.520272:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762731.520307:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8a8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074618752:0@lo:400:kworker.0 00000100:00100000:3.0:1789762731.520319:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762731.520341:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074617984 00000100:00100000:0.0:1789762731.520350:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074617984 00000100:00100000:2.0:1789762731.520367:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762731.520375:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e4d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074619392:0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.520402:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e4ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074619264:0@lo:400:kworker.0 00000100:00100000:3.0:1789762731.520419:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762731.520441:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074618240 00000100:00100000:0.0:1789762731.520448:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074618240 00000100:00100000:2.0:1789762731.520449:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762731.520452:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074619904:0@lo:400:kworker.0 00000100:00100000:0.0:1789762731.520481:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074618112 00000100:00100000:2.0:1789762731.520483:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074619776:0@lo:400:kworker.0 00000100:00100000:0.0:1789762731.520496:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074618112 00000100:00100000:3.0:1789762731.520498:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762731.520533:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074618496 00000100:00100000:2.0:1789762731.520534:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762731.520539:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074618496 00000100:00100000:3.0:1789762731.520541:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1789762731.520552:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c8edd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074618624:0@lo:400:kworker.0 00000100:00100000:0.0:1789762731.520575:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 94 00000100:00100000:3.0:1789762731.520586:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762731.520592:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074617984:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762731.520600:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074618368 00000100:00100000:2.0:1789762731.520605:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074618368 00000100:00100000:3.0:1789762731.520614:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b8df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074619008:0@lo:400:kworker.0 00002000:00000024:0.0:1789762731.520624:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 124190720 pending 0 free 3831496704 avail 3825205248 00000100:00100000:3.0:1789762731.520644:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762731.520645:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074618752 00000100:00100000:2.0:1789762731.520649:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074618752 00000100:00100000:2.0:1789762731.520671:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074619392 00000100:00100000:3.0:1789762731.520672:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e4d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074619520:0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.520674:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074619392 00000100:00100000:0.0:1789762731.520697:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074617984:12345-0@lo:13:osp-pre-0-0.0 Request processed in 111us (516us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762731.520699:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074619264 00000100:00100000:2.0:1789762731.520704:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074619264 00000100:00100000:0.0:1789762731.520713:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 94 00000100:00100000:3.0:1789762731.520714:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762731.520718:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074619136 00000100:00100000:2.0:1789762731.520723:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074619136 00000100:00100000:0.0:1789762731.520750:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 95 00000100:00100000:2.0:1789762731.520753:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074619008 00000100:00100000:2.0:1789762731.520757:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074619008 00000100:00100000:0.0:1789762731.520764:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074618240:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762731.520784:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 223 00000100:00100000:2.0:1789762731.520795:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00002000:00000024:0.0:1789762731.520800:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3843031040 avail 3840933888 00000100:00100000:3.0:1789762731.520800:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074618368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.520810:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074618752:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762731.520876:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074618368:12345-0@lo:400:kworker.0 Request processed in 81us (696us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762731.520884:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074618240:12345-0@lo:13:osp-pre-1-0.0 Request processed in 121us (611us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762731.520885:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074618752:12345-0@lo:400:kworker.0 Request processed in 79us (529us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762731.520890:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 223 00000100:00100000:2.0:1789762731.520908:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00100000:0.0:1789762731.520909:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 95 00000100:00100000:3.0:1789762731.520939:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:3.0:1789762731.520951:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074619392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.520959:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:2.0:1789762731.520974:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074619264:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762731.520976:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762731.520985:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762731.520992:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762731.520997:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762731.521012:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074619392:12345-0@lo:400:kworker.0 Request processed in 61us (590us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762731.521023:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00100000:2.0:1789762731.521028:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074619264:12345-0@lo:400:kworker.0 Request processed in 57us (581us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762731.521042:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d98adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074618240:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762731.521049:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00100000:3.0:1789762731.521058:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:0.0:1789762731.521068:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8a8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074618752:0@lo:400:kworker.0 00000100:00100000:3.0:1789762731.521073:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074619136:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.521078:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:0.0:1789762731.521091:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e4ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074619264:0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.521101:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074619008:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762731.521118:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074619136:12345-0@lo:400:kworker.0 Request processed in 52us (570us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762731.521125:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00100000:3.0:1789762731.521153:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074619904 00000100:00100000:0.0:1789762731.521156:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074619776 00000100:00100000:3.0:1789762731.521159:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074619904 00000100:00100000:0.0:1789762731.521164:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074619776 00000100:00100000:2.0:1789762731.521167:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074619008:12345-0@lo:400:kworker.0 Request processed in 75us (521us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762731.521179:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:3.0:1789762731.521201:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074619648 00000100:00100000:0.0:1789762731.521203:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074619520 00000100:00100000:3.0:1789762731.521208:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074619648 00000100:00100000:0.0:1789762731.521209:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074619520 00000100:00100000:2.0:1789762731.521220:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074618880 00000100:00100000:2.0:1789762731.521225:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074618880 00000100:00100000:0.0:1789762731.521249:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 89 00000100:00100000:2.0:1789762731.521260:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074618624 00000100:00100000:2.0:1789762731.521264:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074618624 00000100:00100000:3.0:1789762731.521267:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 90 00000100:00100000:0.0:1789762731.521273:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074619776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762731.521283:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074619520:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.521288:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:2.0:1789762731.521301:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074618880:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762731.521343:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074619776:12345-0@lo:400:kworker.0 Request processed in 76us (809us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762731.521345:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074619520:12345-0@lo:400:kworker.0 Request processed in 66us (635us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762731.521375:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 90 00000100:00100000:2.0:1789762731.521376:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074618880:12345-0@lo:400:kworker.0 Request processed in 61us (1040us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762731.521384:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 89 00000100:00100000:2.0:1789762731.521400:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000100:00100000:3.0:1789762731.521421:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b8df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074619008:0@lo:400:kworker.0 00000100:00100000:0.0:1789762731.521427:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 162 00000100:00100000:3.0:1789762731.521437:0:3426:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294967464 gen 2 00000100:00100000:0.0:1789762731.521443:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074618624:12345-0@lo:400:kworker.0 00000004:00080000:3.0:1789762731.521445:0:3426:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c36c9ff9500, transno 4294967463 00000004:00080000:3.0:1789762731.521458:0:3426:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c36c9ff9f80, transno 4294967464 00000100:00100000:3.0:1789762731.521466:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e4d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074619520:0@lo:400:kworker.0 00000100:00100000:3.0:1789762731.521490:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b95f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074618880:0@lo:400:kworker.0 00000100:00100000:0.0:1789762731.521500:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074618624:12345-0@lo:400:kworker.0 Request processed in 62us (913us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762731.521510:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 162 00000100:00100000:3.0:1789762731.521514:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e4d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074619392:0@lo:400:kworker.0 00000100:00100000:3.0:1789762731.521541:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074619904:0@lo:400:kworker.0 00000100:00100000:0.0:1789762731.521570:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074619776:0@lo:400:kworker.0 00000100:00100000:3.0:1789762731.521573:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee760000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074618368:0@lo:400:kworker.0 00000100:00100000:3.0:1789762731.521612:0:3426:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967353 gen 2 00000004:00080000:3.0:1789762731.521616:0:3426:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967353 committed transno 4294967353 00000100:00100000:3.0:1789762731.521628:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c8edd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074618624:0@lo:400:kworker.0 00010000:00010000:0.0:1789762731.648085:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789762731.675594:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967486 is committed 00000100:00100000:0.0:1789762731.678081:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762731.862325:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 205109269787 time 205109392892 state 0x0 ref 1 00000100:00100000:3.0:1789762731.862380:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067276160:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762731.862463:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 33554432, mtime: 0x6aad9cab, atime: 0x6aad9ca5, ctime: 0x6aad9cab, blocks: 0x10009 00010000:00010000:3.0:1789762731.862473:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174727 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762731.862492:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=205s v=(0 0) 00010000:00010000:3.0:1789762731.862496:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762731.862502:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174727 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762731.862531:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174727 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762731.876077:0:14225:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967487 is committed 00000100:00100000:2.0:1789762731.878980:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:3.0:1789762732.043118:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967345 is committed 00000100:00100000:2.0:1789762732.062431:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 205309392892 time 205309501844 state 0x0 ref 1 00000100:00100000:2.0:1789762732.062468:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067276288:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762732.062547:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 34603008, mtime: 0x6aad9cab, atime: 0x6aad9ca5, ctime: 0x6aad9cab, blocks: 0x10809 00010000:00010000:2.0:1789762732.062558:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f17472e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.062579:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=205s v=(0 0) 00010000:00010000:2.0:1789762732.062584:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762732.062591:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f17472e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.062620:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f17472e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762732.076107:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967488 is committed 00002000:00010000:2.0:1789762732.076107:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:2.0:1789762732.076115:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 67593 -> 69641 00010000:00010000:2.0:1789762732.076127:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f17472e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.076137:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f17472e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.076144:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f17472e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.076150:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762732.076153:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f17472e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.076161:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f17472e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.076178:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f17472e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.076183:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f17472e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762732.076264:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067276288:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13809us (197284us total) trans 4294967488 rc 0/0 00000100:00100000:2.0:1789762732.076274:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 68 00000100:00100000:2.0:1789762732.076281:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 205309392892 time 205309501844 state 0x2 ref 0 00000100:00100000:0.0:1789762732.078804:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762732.262578:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 205509501844 time 205509644350 state 0x0 ref 1 00000100:00100000:2.0:1789762732.262620:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067276416:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762732.262702:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 35651584, mtime: 0x6aad9cab, atime: 0x6aad9ca5, ctime: 0x6aad9cab, blocks: 0x11009 00010000:00010000:2.0:1789762732.262711:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f174735 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.262729:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=205s v=(0 0) 00010000:00010000:2.0:1789762732.262734:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762732.262740:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f174735 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.262754:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f174735 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762732.278016:0:14225:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967489 is committed 00000100:00100000:0.0:1789762732.281371:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762732.462681:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 205709644350 time 205709746892 state 0x0 ref 1 00000100:00100000:3.0:1789762732.462727:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067276544:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762732.462843:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 36700160, mtime: 0x6aad9cab, atime: 0x6aad9ca5, ctime: 0x6aad9cab, blocks: 0x11809 00010000:00010000:3.0:1789762732.462857:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f17473c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762732.462879:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=205s v=(0 0) 00010000:00010000:3.0:1789762732.462883:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762732.462890:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f17473c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762732.462907:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f17473c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762732.475762:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967490 is committed 00000100:00100000:3.0:1789762732.478455:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762732.478512:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067276672 00000100:00100000:2.0:1789762732.478525:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067276672 00000100:00100000:2.0:1789762732.478579:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 205709644350 time 205709746892 state 0x0 ref 1 00000100:00100000:2.0:1789762732.478596:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 205909746892 time 205709746892 state 0x1 ref 1 00000020:00080000:2.0:1789762732.551886:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 4 00000004:00080000:2.0:1789762732.551985:0:7233:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37c9fc8600 with version 11 00000020:00080000:2.0:1789762732.552004:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 4 cancel update log [0x200000401:0x2:0x0].4: rc = 0 00000004:00080000:0.0:1789762732.552023:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 11 rpc_version 11 00000020:00100000:0.0:1789762732.552067:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762732.552079:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37d98af800 x1876702074620160/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762732.552104:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d98af800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074620160:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762732.552183:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762732.552225:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074620160 00000100:00100000:3.0:1789762732.552235:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074620160 00000100:00100000:0.0:1789762732.552240:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff66ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1789762732.552307:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:3.0:1789762732.552322:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:7224:x1876702074620160:12345-0@lo:1000:osp_up0-1.0 00000020:00080000:2.0:1789762732.552602:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 4 cancel update log [0x240000400:0x2:0x0].4: rc = 0 00000020:00080000:2.0:1789762732.552614:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 4 00000100:00100000:3.0:1789762732.552653:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074620160:12345-0@lo:1000:osp_up0-1.0 Request processed in 335us (469us total) trans 4294967354 rc 0/0 00000100:00100000:3.0:1789762732.552660:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:0.0:1789762732.552708:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d98af800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074620160:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762732.662785:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 205909746892 time 205909845792 state 0x0 ref 1 00000100:00100000:2.0:1789762732.662840:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067276672:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762732.662974:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 37748736, mtime: 0x6aad9cac, atime: 0x6aad9ca5, ctime: 0x6aad9cac, blocks: 0x12009 00010000:00010000:2.0:1789762732.662986:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f174743 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.663008:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=205s v=(0 0) 00010000:00010000:2.0:1789762732.663014:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762732.663022:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f174743 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.663041:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f174743 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762732.672046:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789762732.677250:0:14225:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967491 is committed 00000100:00100000:0.0:1789762732.679869:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762732.862884:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 206109845792 time 206109947027 state 0x0 ref 1 00000100:00100000:3.0:1789762732.862951:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067276800:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762732.863035:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 38797312, mtime: 0x6aad9cac, atime: 0x6aad9ca5, ctime: 0x6aad9cac, blocks: 0x12809 00010000:00010000:3.0:1789762732.863045:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17474a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762732.863080:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=206s v=(0 0) 00010000:00010000:3.0:1789762732.863085:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762732.863092:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17474a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762732.863109:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17474a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762732.882071:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967492 is committed 00000100:00100000:0.0:1789762732.884968:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00080000:0.0:1789762733.063073:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789762733.078563:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967493 is committed 00000100:00100000:2.0:1789762733.081371:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762733.263150:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 206510037729 time 206510211039 state 0x0 ref 1 00000100:00100000:3.0:1789762733.263204:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067277056:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762733.263303:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 40894464, mtime: 0x6aad9cac, atime: 0x6aad9ca5, ctime: 0x6aad9cac, blocks: 0x13809 00010000:00010000:3.0:1789762733.263316:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f174758 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762733.263338:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=206s v=(0 0) 00010000:00010000:3.0:1789762733.263359:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762733.263369:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f174758 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762733.263388:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f174758 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762733.284653:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967494 is committed 00000100:00100000:2.0:1789762733.288993:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762733.463273:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 206710211039 time 206710337282 state 0x0 ref 1 00000100:00100000:2.0:1789762733.463329:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067277184:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762733.463459:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 41943040, mtime: 0x6aad9cac, atime: 0x6aad9ca5, ctime: 0x6aad9cac, blocks: 0x14009 00010000:00010000:2.0:1789762733.463471:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f17475f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762733.463494:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=206s v=(0 0) 00010000:00010000:2.0:1789762733.463500:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762733.463509:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f17475f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762733.463529:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f17475f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762733.480273:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967495 is committed 00000100:00100000:2.0:1789762733.483766:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762733.568205:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074620288:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762733.568315:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762733.568374:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074620288 00000100:00100000:0.0:1789762733.568386:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074620288 00000100:00100000:0.0:1789762733.568471:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:0.0:1789762733.568484:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f63ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074620288:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762733.568533:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762733.568592:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f63ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074620288:12345-0@lo:41:osp-pre-1-0.0 Request processed in 112us (278us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762733.568598:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00100000:2.0:1789762733.568665:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074620288:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762733.663421:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 206910337282 time 206910481947 state 0x0 ref 1 00000100:00100000:2.0:1789762733.663476:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067277312:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762733.663565:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 42991616, mtime: 0x6aad9cad, atime: 0x6aad9ca5, ctime: 0x6aad9cad, blocks: 0x14809 00010000:00010000:2.0:1789762733.663578:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009ea09643/0xdf5a7ac59f174766 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762733.663599:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=206s v=(0 0) 00010000:00010000:2.0:1789762733.663605:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762733.663633:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009ea09643/0xdf5a7ac59f174766 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762733.663652:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009ea09643/0xdf5a7ac59f174766 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762733.677605:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967496 is committed 00000100:00100000:0.0:1789762733.681672:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:0.0:1789762733.696106:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762733.863539:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 207110481947 time 207110603482 state 0x0 ref 1 00000100:00100000:3.0:1789762733.863583:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e4d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067277440:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762733.863672:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 44040192, mtime: 0x6aad9cad, atime: 0x6aad9ca5, ctime: 0x6aad9cad, blocks: 0x15009 00010000:00010000:3.0:1789762733.863682:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17476d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762733.863705:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=207s v=(0 0) 00010000:00010000:3.0:1789762733.863710:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762733.863716:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17476d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762733.863734:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17476d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762733.880151:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967497 is committed 00000100:00100000:2.0:1789762733.884627:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762734.063650:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 207310603482 time 207310716591 state 0x0 ref 1 00000100:00100000:2.0:1789762734.063696:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f63afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067277568:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762734.063772:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 45088768, mtime: 0x6aad9cad, atime: 0x6aad9ca5, ctime: 0x6aad9cad, blocks: 0x15809 00010000:00010000:2.0:1789762734.063783:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174774 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.063806:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=207s v=(0 0) 00010000:00010000:2.0:1789762734.063811:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762734.063819:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174774 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.063838:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174774 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762734.077495:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967498 is committed 00000100:00100000:2.0:1789762734.081143:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762734.263751:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 207510716591 time 207510817629 state 0x0 ref 1 00000100:00100000:3.0:1789762734.263790:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfc5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067277696:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762734.263873:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 46137344, mtime: 0x6aad9cad, atime: 0x6aad9ca5, ctime: 0x6aad9cad, blocks: 0x16009 00010000:00010000:3.0:1789762734.263885:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f17477b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762734.263907:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=207s v=(0 0) 00010000:00010000:3.0:1789762734.263912:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762734.263963:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f17477b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762734.263982:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f17477b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762734.275295:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967499 is committed 00000100:00100000:0.0:1789762734.278213:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762734.463843:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 207710817629 time 207710909879 state 0x0 ref 1 00000100:00100000:0.0:1789762734.463881:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067277824:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762734.463998:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 47185920, mtime: 0x6aad9cad, atime: 0x6aad9ca5, ctime: 0x6aad9cad, blocks: 0x16809 00010000:00010000:0.0:1789762734.464009:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174782 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.464031:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=207s v=(0 0) 00010000:00010000:0.0:1789762734.464037:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.464044:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174782 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.464060:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174782 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762734.476424:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967500 is committed 00000100:00100000:0.0:1789762734.478945:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762734.592226:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789762734.592288:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee760000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074620416:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762734.592401:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762734.592445:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074620416 00000100:00100000:0.0:1789762734.592457:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074620416 00000100:00100000:0.0:1789762734.592521:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:0.0:1789762734.592533:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1748e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3425:x1876702074620416:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762734.592589:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762734.592662:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1748e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3425:x1876702074620416:12345-0@lo:41:osp-pre-0-1.0 Request processed in 131us (260us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762734.592668:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:2.0:1789762734.592704:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee760000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074620416:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762734.663999:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 207910909879 time 207911065087 state 0x0 ref 1 00000100:00100000:3.0:1789762734.664048:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067277952:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762734.664126:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 48234496, mtime: 0x6aad9cae, atime: 0x6aad9ca5, ctime: 0x6aad9cae, blocks: 0x17009 00010000:00010000:3.0:1789762734.664135:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f174789 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762734.664153:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=207s v=(0 0) 00010000:00010000:3.0:1789762734.664157:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762734.664163:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f174789 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762734.664177:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f174789 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762734.676095:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967501 is committed 00000100:00100000:0.0:1789762734.678734:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762734.864110:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 208111065087 time 208111175912 state 0x0 ref 1 00000100:00100000:2.0:1789762734.864167:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067278080:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762734.864272:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 49283072, mtime: 0x6aad9cae, atime: 0x6aad9ca5, ctime: 0x6aad9cae, blocks: 0x17809 00010000:00010000:2.0:1789762734.864284:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f174790 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.864307:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=208s v=(0 0) 00010000:00010000:2.0:1789762734.864324:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762734.864344:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f174790 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.864365:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ecc5aa2/0xdf5a7ac59f174790 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762734.877361:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967502 is committed 00000100:00100000:2.0:1789762734.881290:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762735.064205:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 208311175912 time 208311271554 state 0x0 ref 1 00000100:00100000:2.0:1789762735.064270:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d174ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067278208:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:0.0:1789762735.064293:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762735.064379:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 50331648, mtime: 0x6aad9cae, atime: 0x6aad9ca5, ctime: 0x6aad9cae, blocks: 0x18009 00010000:00010000:2.0:1789762735.064389:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174797 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.064428:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=208s v=(0 0) 00010000:00010000:2.0:1789762735.064433:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762735.064439:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174797 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.064455:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174797 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762735.078035:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967503 is committed 00000100:00100000:2.0:1789762735.081524:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762735.264310:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 208511271554 time 208511370749 state 0x0 ref 1 00000100:00100000:3.0:1789762735.264374:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d174a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067278336:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762735.264469:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 51380224, mtime: 0x6aad9cae, atime: 0x6aad9ca5, ctime: 0x6aad9cae, blocks: 0x18809 00010000:00010000:3.0:1789762735.264499:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f17479e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762735.264518:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=208s v=(0 0) 00010000:00010000:3.0:1789762735.264523:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762735.264529:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f17479e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762735.264551:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f17479e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762735.277341:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967504 is committed 00000100:00100000:2.0:1789762735.280487:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762735.464480:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 208711370749 time 208711545141 state 0x0 ref 1 00000100:00100000:2.0:1789762735.464525:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ca32bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067278464:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762735.464616:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 52428800, mtime: 0x6aad9cae, atime: 0x6aad9ca5, ctime: 0x6aad9cae, blocks: 0x19009 00010000:00010000:2.0:1789762735.464627:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f1747a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.464647:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=208s v=(0 0) 00010000:00010000:2.0:1789762735.464651:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762735.464657:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f1747a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.464673:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3cc0846/0xdf5a7ac59f1747a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762735.479408:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967505 is committed 00000100:00100000:2.0:1789762735.482433:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762735.560120:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762735.560175:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762735.560185:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067278720 00000100:00100000:2.0:1789762735.560200:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067278720 00000100:00100000:0.0:1789762735.560212:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762735.560225:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067278848 00000100:00100000:3.0:1789762735.560232:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067278848 00000100:00100000:0.0:1789762735.560260:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762735.560274:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 307 00000100:00100000:2.0:1789762735.560289:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1551450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067278720:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762735.560293:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067278976 00000100:00100000:3.0:1789762735.560298:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067278976 00000100:00100000:0.0:1789762735.560305:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762735.560342:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067279360 00000100:00100000:3.0:1789762735.560345:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067279360 00000100:00100000:3.0:1789762735.560380:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 229 00000100:00100000:0.0:1789762735.560386:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762735.560395:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067278848:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762735.560405:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1551450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067278720:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 119us (300us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762735.560412:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 307 00000100:00100000:3.0:1789762735.560439:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067278848:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 49us (264us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762735.560448:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 229 00000100:00100000:3.0:1789762735.560466:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067279232 00000100:00100000:3.0:1789762735.560470:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067279232 00000100:00100000:2.0:1789762735.560480:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 230 00000100:00100000:2.0:1789762735.560493:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067278976:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762735.560504:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 232 00000100:00100000:3.0:1789762735.560517:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067279232:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762735.560528:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067278976:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 39us (269us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762735.560535:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 230 00000100:00100000:3.0:1789762735.560552:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067279232:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 36us (168us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762735.560578:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 232 00000001:00080000:3.0:1789762735.627463:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967354 is committed 00000100:00100000:3.0:1789762735.664553:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 208911545141 time 208911624037 state 0x0 ref 1 00000100:00100000:3.0:1789762735.664587:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7832300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067278592:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762735.664664:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 53477376, mtime: 0x6aad9caf, atime: 0x6aad9ca5, ctime: 0x6aad9caf, blocks: 0x1980b 00010000:00010000:3.0:1789762735.664673:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f1747ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762735.664692:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=208s v=(0 0) 00010000:00010000:3.0:1789762735.664697:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762735.664703:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f1747ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762735.664719:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f1747ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762735.681971:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 53477376 -> 54525952 00000001:00080000:3.0:1789762735.681972:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967506 is committed 00002000:00010000:2.0:1789762735.681981:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 104459 -> 106507 00010000:00010000:2.0:1789762735.682003:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f1747ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.682021:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f1747ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.682042:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f1747ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.682066:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762735.682073:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f1747ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.682087:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f1747ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.682127:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f1747ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.682133:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f1747ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762735.682254:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7832300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067278592:12345-192.168.203.37@tcp:4:dd.500 Request processed in 17675us (199818us total) trans 4294967506 rc 0/0 00000100:00100000:2.0:1789762735.682266:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 86 00000100:00100000:2.0:1789762735.682276:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 208911545141 time 208911624037 state 0x2 ref 0 00000100:00100000:3.0:1789762735.685082:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762735.685128:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067279488 00000100:00100000:0.0:1789762735.685142:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067279488 00000100:00100000:0.0:1789762735.685207:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 208911545141 time 208911624037 state 0x0 ref 1 00000100:00100000:0.0:1789762735.685581:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 209111624037 time 208911624037 state 0x1 ref 1 00010000:00010000:0.0:1789762735.745042:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762735.864621:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 209111624037 time 209111688364 state 0x0 ref 1 00000100:00100000:0.0:1789762735.864675:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee760000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067279488:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762735.864753:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 54525952, mtime: 0x6aad9caf, atime: 0x6aad9ca5, ctime: 0x6aad9caf, blocks: 0x1a00b 00010000:00010000:0.0:1789762735.864762:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f1747b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.864780:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=209s v=(0 0) 00010000:00010000:0.0:1789762735.864784:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762735.864790:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f1747b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.864806:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f1747b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762735.883824:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967507 is committed 00000100:00100000:3.0:1789762735.887150:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762736.064709:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 209311688364 time 209311780381 state 0x0 ref 1 00000100:00100000:3.0:1789762736.064741:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b95f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067279616:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762736.064799:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 55574528, mtime: 0x6aad9caf, atime: 0x6aad9ca5, ctime: 0x6aad9caf, blocks: 0x1a80b 00010000:00010000:3.0:1789762736.064820:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1747ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762736.064836:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=209s v=(0 0) 00010000:00010000:3.0:1789762736.064841:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762736.064848:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1747ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762736.064862:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f1747ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762736.077891:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967508 is committed 00000100:00100000:3.0:1789762736.080813:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762736.080868:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067279744 00000100:00100000:2.0:1789762736.080877:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067279744 00000100:00100000:2.0:1789762736.080949:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 209311688364 time 209311780381 state 0x0 ref 1 00000100:00100000:2.0:1789762736.080959:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 209511780381 time 209311780381 state 0x1 ref 1 00000100:00100000:0.0:1789762736.264823:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 209511780381 time 209511895314 state 0x0 ref 1 00000100:00100000:0.0:1789762736.264852:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067279744:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762736.264913:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 56623104, mtime: 0x6aad9caf, atime: 0x6aad9ca5, ctime: 0x6aad9caf, blocks: 0x1b00b 00010000:00010000:0.0:1789762736.264954:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051269a18/0xdf5a7ac59f1747c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.264968:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=209s v=(0 0) 00010000:00010000:0.0:1789762736.264972:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762736.264977:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051269a18/0xdf5a7ac59f1747c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.265008:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051269a18/0xdf5a7ac59f1747c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762736.276951:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967509 is committed 00000100:00100000:3.0:1789762736.280214:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762736.465025:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 209711895314 time 209712090931 state 0x0 ref 1 00000100:00100000:2.0:1789762736.465070:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067279872:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762736.465165:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 57671680, mtime: 0x6aad9caf, atime: 0x6aad9ca5, ctime: 0x6aad9caf, blocks: 0x1b80b 00010000:00010000:2.0:1789762736.465177:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f1747c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762736.465199:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=209s v=(0 0) 00010000:00010000:2.0:1789762736.465205:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762736.465213:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f1747c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762736.465233:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ecce050/0xdf5a7ac59f1747c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762736.490242:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967510 is committed 00000100:00100000:3.0:1789762736.494977:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762736.640220:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c8173800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074620800:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762736.640361:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762736.640485:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074621184:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762736.640539:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6096300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074620544:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.640556:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074620800 00000100:00100000:2.0:1789762736.640566:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762736.640572:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074620800 00000100:00100000:2.0:1789762736.640620:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074621696:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762736.640660:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762736.640667:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074621184 00000100:00100000:2.0:1789762736.640677:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762736.640680:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074621184 00000100:00100000:0.0:1789762736.640747:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074621696 00000100:00100000:0.0:1789762736.640760:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074621696 00000100:00100000:2.0:1789762736.640775:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb930380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074622208:0@lo:400:kworker.0 00000100:00100000:3.0:1789762736.640781:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6095c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074621056:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.640819:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 98 00000100:00100000:0.0:1789762736.640850:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074620800:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762736.640881:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762736.640882:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:0.0:1789762736.640973:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762736.641006:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee70ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074621568:0@lo:400:kworker.0 00000100:00100000:2.0:1789762736.641010:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:2.0:1789762736.641031:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074620672:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762736.641096:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074620800:12345-0@lo:13:osp-pre-1-1.0 Request processed in 253us (737us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762736.641122:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762736.641124:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762736.641125:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 98 00000100:00100000:2.0:1789762736.641156:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee70d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074621312:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.641171:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074620672 00000100:00100000:0.0:1789762736.641180:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074620672 00000100:00100000:3.0:1789762736.641199:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee70c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074622080:0@lo:400:kworker.0 00000100:00100000:2.0:1789762736.641211:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762736.641236:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 99 00000100:00100000:0.0:1789762736.641253:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074621184:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789762736.641273:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762736.641280:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee70ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074621824:0@lo:400:kworker.0 00000100:00100000:3.0:1789762736.641292:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762736.641380:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762736.641396:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074621184:12345-0@lo:13:osp-pre-1-0.0 Request processed in 144us (836us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762736.641414:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074622080 00000100:00100000:2.0:1789762736.641419:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff615c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074622336:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.641420:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 99 00000100:00100000:3.0:1789762736.641431:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074622080 00000100:00100000:0.0:1789762736.641467:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 100 00000100:00100000:2.0:1789762736.641491:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762736.641498:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 233 00000100:00100000:0.0:1789762736.641506:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074621696:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762736.641527:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074620928:12345-0@lo:400:kworker.0 00002000:00000024:0.0:1789762736.641530:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 166789120 pending 0 free 3810525184 avail 3798990848 00000100:00100000:2.0:1789762736.641570:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074621184:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762736.641601:0:3426:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294967345 gen 2 00000004:00080000:2.0:1789762736.641609:0:3426:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c36c9ffc380, transno 4294967344 00000100:00100000:0.0:1789762736.641621:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074621696:12345-0@lo:13:osp-pre-0-0.0 Request processed in 126us (946us total) trans 0 rc 0/0 00000004:00080000:2.0:1789762736.641624:0:3426:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c36c9ffc700, transno 4294967345 00000100:00100000:0.0:1789762736.641638:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 100 00000100:00100000:2.0:1789762736.641650:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c8173800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074620800:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762736.641672:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074620928:12345-0@lo:400:kworker.0 Request processed in 152us (1202us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762736.641687:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 101 00000100:00100000:3.0:1789762736.641699:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 233 00000100:00100000:2.0:1789762736.641721:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074621696:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762736.641723:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074620672:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789762736.641751:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 166789120 pending 0 free 3810525184 avail 3798990848 00000100:00100000:3.0:1789762736.641754:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:2.0:1789762736.641784:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074622208 00000100:00100000:3.0:1789762736.641785:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfcea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074621568:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762736.641798:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074622208 00000100:00100000:0.0:1789762736.641825:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074620672:12345-0@lo:13:osp-pre-0-1.0 Request processed in 115us (714us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762736.641838:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 101 00000100:00080000:0.0:1789762736.641878:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762736.641886:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074622464 00000100:00080000:0.0:1789762736.641887:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762736.641897:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074622464 00000100:00080000:0.0:1789762736.641900:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762736.641911:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762736.641967:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfcea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074621568:12345-0@lo:400:kworker.0 Request processed in 145us (811us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762736.641981:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074622336 00000100:00100000:2.0:1789762736.641988:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074622336 00000100:00100000:3.0:1789762736.641991:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00000100:00100000:0.0:1789762736.641994:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074621440 00000100:00100000:0.0:1789762736.642004:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074621440 00000100:00100000:2.0:1789762736.642028:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 93 00000100:00100000:2.0:1789762736.642056:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074622208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762736.642058:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:0.0:1789762736.642067:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074621056 00000100:00100000:0.0:1789762736.642075:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074621056 00000100:00100000:3.0:1789762736.642092:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074621824:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.642117:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:0.0:1789762736.642135:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074621440:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762736.642166:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074622208:12345-0@lo:400:kworker.0 Request processed in 123us (1294us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762736.642186:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 93 00000100:00100000:3.0:1789762736.642220:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074621824:12345-0@lo:400:kworker.0 Request processed in 135us (839us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762736.642229:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 94 00000100:00100000:0.0:1789762736.642234:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074621440:12345-0@lo:400:kworker.0 Request processed in 100us (1577us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762736.642255:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074622464:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.642265:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:3.0:1789762736.642265:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00100000:2.0:1789762736.642362:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074622464:12345-0@lo:400:kworker.0 Request processed in 111us (1356us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762736.642374:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 95 00000100:00100000:2.0:1789762736.642386:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 94 00000100:00100000:3.0:1789762736.642396:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074621568:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.642397:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3beea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074622336:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762736.642454:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074622080:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.642489:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3beea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074622336:12345-0@lo:400:kworker.0 Request processed in 100us (998us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762736.642512:0:3425:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967354 gen 2 00000100:00100000:0.0:1789762736.642519:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 95 00000004:00080000:3.0:1789762736.642523:0:3425:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967354 committed transno 4294967354 00000100:00100000:3.0:1789762736.642558:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6095c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074621056:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.642618:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074620672:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762736.642646:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074621312:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.642681:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074621824:0@lo:400:kworker.0 00000100:00100000:3.0:1789762736.642721:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074621952:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.642727:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff615c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074622336:0@lo:400:kworker.0 00000100:00100000:3.0:1789762736.642766:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff617480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074622464:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.642774:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb930380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074622208:0@lo:400:kworker.0 00000100:00100000:3.0:1789762736.642810:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6095500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074620928:0@lo:400:kworker.0 00000100:00100000:3.0:1789762736.642856:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074621440:0@lo:400:kworker.0 00000001:00080000:3.0:1789762736.684165:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967511 is committed 00000100:00100000:0.0:1789762736.687894:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:2.0:1789762736.768067:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762736.865306:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 210112179591 time 210112373757 state 0x0 ref 1 00000100:00100000:2.0:1789762736.865365:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067280128:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762736.865434:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 59768832, mtime: 0x6aad9cb0, atime: 0x6aad9ca5, ctime: 0x6aad9cb0, blocks: 0x1c80b 00010000:00010000:2.0:1789762736.865446:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f1747d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762736.865467:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=210s v=(0 0) 00010000:00010000:2.0:1789762736.865473:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762736.865480:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f1747d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762736.865499:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f1747d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762736.884938:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967512 is committed 00000100:00100000:3.0:1789762736.888734:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762737.065476:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 210312373757 time 210312543127 state 0x0 ref 1 00000100:00100000:2.0:1789762737.065545:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067280256:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:0.0:1789762737.065592:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762737.065620:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 60817408, mtime: 0x6aad9cb0, atime: 0x6aad9ca5, ctime: 0x6aad9cb0, blocks: 0x1d00b 00010000:00010000:2.0:1789762737.065634:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000926cfe09/0xdf5a7ac59f1747dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.065656:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=210s v=(0 0) 00010000:00010000:2.0:1789762737.065662:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762737.065671:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000926cfe09/0xdf5a7ac59f1747dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.065710:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000926cfe09/0xdf5a7ac59f1747dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762737.088866:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967513 is committed 00000100:00100000:3.0:1789762737.093293:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762737.265715:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 210512543127 time 210512764365 state 0x0 ref 1 00000100:00100000:0.0:1789762737.265760:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6096300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067280384:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762737.265833:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 61865984, mtime: 0x6aad9cb0, atime: 0x6aad9ca5, ctime: 0x6aad9cb0, blocks: 0x1d80b 00010000:00010000:0.0:1789762737.265845:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1747e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.265866:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=210s v=(0 0) 00010000:00010000:0.0:1789762737.265873:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762737.265881:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1747e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.265917:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1747e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762737.289635:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967514 is committed 00000100:00100000:2.0:1789762737.294398:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762737.465963:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 210712764365 time 210713027591 state 0x0 ref 1 00000100:00100000:3.0:1789762737.466008:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067280512:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762737.466095:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 62914560, mtime: 0x6aad9cb0, atime: 0x6aad9ca5, ctime: 0x6aad9cb0, blocks: 0x1e00b 00010000:00010000:3.0:1789762737.466109:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1747eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762737.466129:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=210s v=(0 0) 00010000:00010000:3.0:1789762737.466136:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762737.466144:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1747eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762737.466162:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1747eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762737.488554:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967515 is committed 00000100:00100000:2.0:1789762737.493050:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762737.666215:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 210913027591 time 210913285831 state 0x0 ref 1 00000100:00100000:0.0:1789762737.666269:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067280640:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762737.666390:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 63963136, mtime: 0x6aad9cb1, atime: 0x6aad9ca5, ctime: 0x6aad9cb1, blocks: 0x1e80b 00010000:00010000:0.0:1789762737.666402:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1747f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.666421:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=210s v=(0 0) 00010000:00010000:0.0:1789762737.666428:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762737.666435:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1747f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.666455:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1747f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762737.690980:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967516 is committed 00000100:00100000:0.0:1789762737.694590:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:0.0:1789762737.792047:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762737.866414:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 211113285831 time 211113481959 state 0x0 ref 1 00000100:00100000:2.0:1789762737.866458:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff617480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067280768:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762737.866528:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 65011712, mtime: 0x6aad9cb1, atime: 0x6aad9ca5, ctime: 0x6aad9cb1, blocks: 0x1f00b 00010000:00010000:2.0:1789762737.866540:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1747f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.866560:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=211s v=(0 0) 00010000:00010000:2.0:1789762737.866566:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762737.866574:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1747f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.866593:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f1747f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762737.888373:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967517 is committed 00000100:00100000:0.0:1789762737.891901:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762738.066516:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 211313481959 time 211313585591 state 0x0 ref 1 00000100:00100000:2.0:1789762738.066596:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067280896:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762738.066668:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 66060288, mtime: 0x6aad9cb1, atime: 0x6aad9ca5, ctime: 0x6aad9cb1, blocks: 0x1f80b 00010000:00010000:2.0:1789762738.066680:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000926cfe09/0xdf5a7ac59f174800 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.066716:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=211s v=(0 0) 00010000:00010000:2.0:1789762738.066724:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762738.066732:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000926cfe09/0xdf5a7ac59f174800 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.066754:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000926cfe09/0xdf5a7ac59f174800 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762738.087289:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967518 is committed 00000100:00100000:0.0:1789762738.091249:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762738.266635:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 211513585591 time 211513691214 state 0x0 ref 1 00000100:00100000:2.0:1789762738.266677:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ebaa0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067281024:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762738.266759:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 67108864, mtime: 0x6aad9cb1, atime: 0x6aad9ca5, ctime: 0x6aad9cb1, blocks: 0x2000b 00010000:00010000:2.0:1789762738.266772:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174807 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.266794:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=211s v=(0 0) 00010000:00010000:2.0:1789762738.266800:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762738.266809:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174807 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.266853:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174807 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762738.279598:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967519 is committed 00002000:00010000:2.0:1789762738.279673:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 67108864 -> 68157440 00002000:00010000:2.0:1789762738.279680:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 131083 -> 133131 00010000:00010000:2.0:1789762738.279692:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174807 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.279706:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174807 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.279719:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174807 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.279728:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cfc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762738.279733:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174807 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.279745:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174807 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.279763:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174807 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.279773:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004be54199/0xdf5a7ac59f174807 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762738.279854:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067281024:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13194us (188608us total) trans 4294967519 rc 0/0 00000100:00100000:2.0:1789762738.279866:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 99 00000100:00100000:2.0:1789762738.279874:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 211513585591 time 211513691214 state 0x2 ref 0 00000100:00100000:0.0:1789762738.282853:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762738.466719:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 211713691214 time 211713788474 state 0x0 ref 1 00000100:00100000:0.0:1789762738.466785:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a34380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067281152:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762738.466867:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 68157440, mtime: 0x6aad9cb1, atime: 0x6aad9ca5, ctime: 0x6aad9cb1, blocks: 0x2080b 00010000:00010000:0.0:1789762738.466906:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17480e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.466975:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=211s v=(0 0) 00010000:00010000:0.0:1789762738.467001:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762738.467010:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17480e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.467030:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17480e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762738.483607:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967520 is committed 00002000:00010000:0.0:1789762738.483655:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 68157440 -> 69206016 00002000:00010000:0.0:1789762738.483661:0:14171:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb mtime from disk: 1789762737 -> 1789762738 00002000:00010000:0.0:1789762738.483665:0:14171:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb ctime from disk: 1789762737 -> 1789762738 00002000:00010000:0.0:1789762738.483669:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 133131 -> 135179 00010000:00010000:0.0:1789762738.483684:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17480e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.483701:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17480e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.483710:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17480e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.483717:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762738.483722:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17480e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.483739:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17480e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.483759:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17480e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.483766:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17480e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762738.483845:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a34380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067281152:12345-192.168.203.37@tcp:4:dd.500 Request processed in 17100us (200990us total) trans 4294967520 rc 0/0 00000100:00100000:0.0:1789762738.483859:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 100 00000100:00100000:0.0:1789762738.483869:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 211713691214 time 211713788474 state 0x2 ref 0 00000100:00100000:2.0:1789762738.486737:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762738.666832:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 211913788474 time 211913901469 state 0x0 ref 1 00000100:00100000:0.0:1789762738.666866:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067281280:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762738.666955:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 69206016, mtime: 0x6aad9cb2, atime: 0x6aad9ca5, ctime: 0x6aad9cb2, blocks: 0x2100b 00010000:00010000:0.0:1789762738.666966:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174815 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.666984:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=211s v=(0 0) 00010000:00010000:0.0:1789762738.666990:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762738.666996:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174815 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.667033:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174815 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762738.681040:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 69206016 -> 70254592 00002000:00010000:0.0:1789762738.681047:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 135179 -> 137227 00000001:00080000:2.0:1789762738.681048:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967521 is committed 00010000:00010000:0.0:1789762738.681061:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174815 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.681075:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174815 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.681085:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174815 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.681093:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762738.681098:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174815 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.681107:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174815 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.681126:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174815 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.681134:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174815 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762738.681219:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067281280:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14365us (194482us total) trans 4294967521 rc 0/0 00000100:00100000:0.0:1789762738.681230:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 101 00000100:00100000:0.0:1789762738.681238:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 211913788474 time 211913901469 state 0x2 ref 0 00000100:00100000:2.0:1789762738.683537:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067281408 00000100:00100000:2.0:1789762738.683546:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067281408 00000100:00100000:2.0:1789762738.683581:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 211913788474 time 211913901469 state 0x0 ref 1 00000100:00100000:2.0:1789762738.683592:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212113901469 time 211913901469 state 0x1 ref 1 00000100:00100000:0.0:1789762738.688180:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074622592 00000100:00100000:0.0:1789762738.688188:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074622592 00000100:00100000:0.0:1789762738.688247:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:0.0:1789762738.688259:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074622592:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762738.688292:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762738.688372:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074622592:12345-0@lo:41:osp-pre-1-0.0 Request processed in 117us (240us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762738.688383:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00100000:3.0:1789762738.866963:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212113901469 time 212114034961 state 0x0 ref 1 00000100:00100000:3.0:1789762738.866992:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1771f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067281408:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762738.867043:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 70254592, mtime: 0x6aad9cb2, atime: 0x6aad9ca5, ctime: 0x6aad9cb2, blocks: 0x2180b 00010000:00010000:3.0:1789762738.867051:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17481c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762738.867078:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=212s v=(0 0) 00010000:00010000:3.0:1789762738.867082:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762738.867087:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17481c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762738.867100:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17481c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762738.879905:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 70254592 -> 71303168 00000001:00080000:2.0:1789762738.879907:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967522 is committed 00002000:00010000:0.0:1789762738.879912:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 137227 -> 139275 00010000:00010000:0.0:1789762738.879945:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17481c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.879956:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17481c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.879963:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17481c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.879969:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762738.879978:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17481c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.879986:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17481c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.880000:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17481c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.880006:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f17481c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762738.880067:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1771f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067281408:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13087us (196595us total) trans 4294967522 rc 0/0 00000100:00100000:0.0:1789762738.880074:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 102 00000100:00100000:0.0:1789762738.880081:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212113901469 time 212114034961 state 0x2 ref 0 00000100:00100000:0.0:1789762738.882372:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067281536 00000100:00100000:0.0:1789762738.882380:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067281536 00000100:00100000:0.0:1789762738.882411:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212113901469 time 212114034961 state 0x0 ref 1 00000100:00100000:0.0:1789762738.882420:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212314034961 time 212114034961 state 0x1 ref 1 00000100:00100000:0.0:1789762739.067030:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212314034961 time 212314101044 state 0x0 ref 1 00000100:00100000:0.0:1789762739.067075:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067281536:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762739.067128:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 71303168, mtime: 0x6aad9cb2, atime: 0x6aad9ca5, ctime: 0x6aad9cb2, blocks: 0x2200b 00010000:00010000:0.0:1789762739.067136:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174823 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.067164:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=212s v=(0 0) 00010000:00010000:0.0:1789762739.067168:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762739.067174:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174823 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.067185:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174823 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762739.081935:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 71303168 -> 72351744 00002000:00010000:0.0:1789762739.081941:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 139275 -> 141323 00010000:00010000:0.0:1789762739.081951:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174823 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.081960:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174823 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.081966:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174823 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.081971:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762739.081975:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174823 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.081982:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174823 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.082007:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174823 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.082013:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174823 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762739.082072:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067281536:12345-192.168.203.37@tcp:4:dd.500 Request processed in 15027us (199748us total) trans 4294967523 rc 0/0 00000100:00100000:0.0:1789762739.082078:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 103 00000100:00100000:0.0:1789762739.082083:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212314034961 time 212314101044 state 0x2 ref 0 00000100:00100000:2.0:1789762739.084357:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762739.084406:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067281664 00000100:00100000:0.0:1789762739.084416:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067281664 00000100:00100000:0.0:1789762739.084454:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212314034961 time 212314101044 state 0x0 ref 1 00000100:00100000:0.0:1789762739.084463:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212514101044 time 212314101044 state 0x1 ref 1 00000001:00080000:3.0:1789762739.280893:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967524 is committed 00002000:00010000:2.0:1789762739.280904:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 72351744 -> 73400320 00002000:00010000:2.0:1789762739.280911:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 141323 -> 143371 00010000:00010000:2.0:1789762739.280945:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f17482a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.280956:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f17482a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.280963:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f17482a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.280968:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762739.280979:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f17482a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.280987:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f17482a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.281003:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f17482a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.281008:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f17482a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762739.281079:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3beed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067281664:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13932us (196721us total) trans 4294967524 rc 0/0 00000100:00100000:2.0:1789762739.281088:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 104 00000100:00100000:2.0:1789762739.281095:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212514101044 time 212514192248 state 0x2 ref 0 00000100:00100000:2.0:1789762739.283403:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762739.283456:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067281792 00000100:00100000:0.0:1789762739.283465:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067281792 00000100:00100000:0.0:1789762739.283501:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212514101044 time 212514192248 state 0x0 ref 1 00000100:00100000:0.0:1789762739.283510:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212714192248 time 212514192248 state 0x1 ref 1 00000001:00080000:3.0:1789762739.483664:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967525 is committed 00002000:00010000:2.0:1789762739.483666:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 73400320 -> 74448896 00002000:00010000:2.0:1789762739.483674:0:8058:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb mtime from disk: 1789762738 -> 1789762739 00002000:00010000:2.0:1789762739.483677:0:8058:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb ctime from disk: 1789762738 -> 1789762739 00002000:00010000:2.0:1789762739.483680:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 143371 -> 145419 00010000:00010000:2.0:1789762739.483693:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174831 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.483704:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174831 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.483723:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174831 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.483728:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762739.483732:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174831 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.483738:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174831 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.483752:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174831 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.483757:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174831 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762739.483825:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067281792:12345-192.168.203.37@tcp:4:dd.500 Request processed in 16572us (200422us total) trans 4294967525 rc 0/0 00000100:00100000:2.0:1789762739.483833:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 105 00000100:00100000:2.0:1789762739.483838:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212714192248 time 212714300861 state 0x2 ref 0 00000100:00100000:0.0:1789762739.486260:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762739.667407:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212914300861 time 212914479688 state 0x0 ref 1 00000100:00100000:2.0:1789762739.667437:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067281920:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762739.667486:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 74448896, mtime: 0x6aad9cb3, atime: 0x6aad9ca5, ctime: 0x6aad9cb3, blocks: 0x2380b 00010000:00010000:2.0:1789762739.667506:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174838 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.667519:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=212s v=(0 0) 00010000:00010000:2.0:1789762739.667523:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762739.667527:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174838 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.667540:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174838 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762739.681101:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:0.0:1789762739.681107:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 145419 -> 147467 00010000:00010000:0.0:1789762739.681117:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174838 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.681132:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174838 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.681139:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174838 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.681145:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cf800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762739.681148:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174838 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.681154:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174838 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.681168:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174838 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.681173:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b045ca27/0xdf5a7ac59f174838 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762739.681227:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067281920:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13805us (194970us total) trans 4294967526 rc 0/0 00000100:00100000:0.0:1789762739.681234:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 106 00000100:00100000:0.0:1789762739.681240:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212914300861 time 212914479688 state 0x2 ref 0 00000100:00100000:0.0:1789762739.683441:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762739.683501:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067282048 00000100:00100000:2.0:1789762739.683510:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067282048 00000100:00100000:2.0:1789762739.683541:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 212914300861 time 212914479688 state 0x0 ref 1 00000100:00100000:2.0:1789762739.683550:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213114479688 time 212914479688 state 0x1 ref 1 00000100:00100000:0.0:1789762739.712251:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074622720 00000100:00100000:0.0:1789762739.712259:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074622720 00000100:00100000:0.0:1789762739.712321:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:0.0:1789762739.712329:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074622720:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762739.712358:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762739.712409:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074622720:12345-0@lo:41:osp-pre-0-1.0 Request processed in 82us (202us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762739.712415:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00010000:00010000:0.0:1789762739.840112:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789762739.881947:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967527 is committed 00002000:00010000:0.0:1789762739.881980:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 75497472 -> 76546048 00002000:00010000:0.0:1789762739.881986:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 147467 -> 149515 00010000:00010000:0.0:1789762739.881998:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17483f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.882008:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17483f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.882015:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17483f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.882020:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762739.882023:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17483f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.882030:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17483f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.882056:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17483f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.882062:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17483f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762739.882154:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067282048:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14586us (198709us total) trans 4294967527 rc 0/0 00000100:00100000:0.0:1789762739.882164:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 107 00000100:00100000:0.0:1789762739.882172:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213114479688 time 213114613094 state 0x2 ref 0 00000100:00100000:0.0:1789762739.884447:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762739.884506:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067282176 00000100:00100000:2.0:1789762739.884517:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067282176 00000100:00100000:2.0:1789762739.884564:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213114479688 time 213114613094 state 0x0 ref 1 00000100:00100000:2.0:1789762739.884577:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213314613094 time 213114613094 state 0x1 ref 1 00000100:00100000:0.0:1789762740.067662:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213314613094 time 213314728628 state 0x0 ref 1 00000100:00100000:0.0:1789762740.067697:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067282176:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762740.067781:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 76546048, mtime: 0x6aad9cb3, atime: 0x6aad9ca5, ctime: 0x6aad9cb3, blocks: 0x2480b 00010000:00010000:0.0:1789762740.067792:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f174846 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.067811:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=213s v=(0 0) 00010000:00010000:0.0:1789762740.067827:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762740.067833:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f174846 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.067847:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f174846 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762740.082731:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967528 is committed 00002000:00010000:0.0:1789762740.082758:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 76546048 -> 77594624 00002000:00010000:0.0:1789762740.082765:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 149515 -> 151563 00010000:00010000:0.0:1789762740.082780:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f174846 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.082792:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f174846 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.082800:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f174846 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.082805:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762740.082810:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f174846 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.082818:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f174846 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.082839:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f174846 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.082844:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f174846 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762740.082978:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067282176:12345-192.168.203.37@tcp:4:dd.500 Request processed in 15281us (198516us total) trans 4294967528 rc 0/0 00000100:00100000:0.0:1789762740.083000:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 108 00000100:00100000:0.0:1789762740.083010:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213314613094 time 213314728628 state 0x2 ref 0 00000100:00100000:2.0:1789762740.085591:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762740.085672:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067282304 00000100:00100000:0.0:1789762740.085685:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067282304 00000100:00100000:0.0:1789762740.085742:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213314613094 time 213314728628 state 0x0 ref 1 00000100:00100000:0.0:1789762740.085754:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213514728628 time 213314728628 state 0x1 ref 1 00000100:00100000:0.0:1789762740.267742:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213514728628 time 213514809441 state 0x0 ref 1 00000100:00100000:0.0:1789762740.267787:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067282304:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762740.267868:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 77594624, mtime: 0x6aad9cb3, atime: 0x6aad9ca5, ctime: 0x6aad9cb3, blocks: 0x2500b 00010000:00010000:0.0:1789762740.267879:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f17484d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.267899:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=213s v=(0 0) 00010000:00010000:0.0:1789762740.267904:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762740.267913:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f17484d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.267955:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f17484d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762740.281864:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967529 is committed 00002000:00010000:0.0:1789762740.281897:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:0.0:1789762740.281903:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 151563 -> 153611 00010000:00010000:0.0:1789762740.281952:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f17484d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.281975:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f17484d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.281987:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f17484d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.281997:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762740.282004:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f17484d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.282016:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f17484d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.282041:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f17484d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.282051:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f17484d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762740.282152:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067282304:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14380us (196557us total) trans 4294967529 rc 0/0 00000100:00100000:0.0:1789762740.282164:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 109 00000100:00100000:0.0:1789762740.282175:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213514728628 time 213514809441 state 0x2 ref 0 00000100:00100000:0.0:1789762740.284579:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067282432 00000100:00100000:0.0:1789762740.284597:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067282432 00000100:00100000:0.0:1789762740.284651:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213514728628 time 213514809441 state 0x0 ref 1 00000100:00100000:0.0:1789762740.284668:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213714809441 time 213514809441 state 0x1 ref 1 00000001:00080000:3.0:1789762740.485217:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967530 is committed 00002000:00010000:0.0:1789762740.485266:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:0.0:1789762740.485272:0:8057:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb mtime from disk: 1789762739 -> 1789762740 00002000:00010000:0.0:1789762740.485276:0:8057:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb ctime from disk: 1789762739 -> 1789762740 00002000:00010000:0.0:1789762740.485279:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 153611 -> 155659 00010000:00010000:0.0:1789762740.485307:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fb25706a/0xdf5a7ac59f174854 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.485332:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fb25706a/0xdf5a7ac59f174854 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.485343:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fb25706a/0xdf5a7ac59f174854 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.485352:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762740.485358:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fb25706a/0xdf5a7ac59f174854 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.485369:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fb25706a/0xdf5a7ac59f174854 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.485393:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fb25706a/0xdf5a7ac59f174854 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.485418:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb25706a/0xdf5a7ac59f174854 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762740.485541:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067282432:12345-192.168.203.37@tcp:4:dd.500 Request processed in 17647us (201008us total) trans 4294967530 rc 0/0 00000100:00100000:0.0:1789762740.485556:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 110 00000100:00100000:0.0:1789762740.485568:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213714809441 time 213714933439 state 0x2 ref 0 00000100:00100000:0.0:1789762740.488152:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067282560 00000100:00100000:0.0:1789762740.488176:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067282560 00000100:00100000:0.0:1789762740.488227:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213714809441 time 213714933439 state 0x0 ref 1 00000100:00100000:0.0:1789762740.488242:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213914933439 time 213714933439 state 0x1 ref 1 00000100:00100000:2.0:1789762740.668020:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213914933439 time 213915071084 state 0x0 ref 1 00000100:00100000:2.0:1789762740.668067:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067282560:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762740.668139:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 79691776, mtime: 0x6aad9cb4, atime: 0x6aad9ca5, ctime: 0x6aad9cb4, blocks: 0x2600b 00010000:00010000:2.0:1789762740.668151:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ff23cf49/0xdf5a7ac59f17485b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762740.668198:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=213s v=(0 0) 00010000:00010000:2.0:1789762740.668202:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762740.668206:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff23cf49/0xdf5a7ac59f17485b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762740.668218:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff23cf49/0xdf5a7ac59f17485b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762740.679673:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762740.679712:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762740.679745:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762740.679847:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762740.679974:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762740.680010:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067283200 00000100:00100000:0.0:1789762740.680019:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067283200 00000100:00100000:0.0:1789762740.680118:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067282816 00000100:00100000:0.0:1789762740.680123:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067282816 00000100:00100000:0.0:1789762740.680190:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 239 00000100:00100000:0.0:1789762740.680208:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdddc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067283200:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762740.680336:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdddc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067283200:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 131us (585us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762740.680360:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 239 00000100:00100000:0.0:1789762740.680413:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 240 00000100:00100000:0.0:1789762740.680431:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdddfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067282816:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762740.680485:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdddfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067282816:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 59us (639us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762740.680496:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 240 00000100:00100000:2.0:1789762740.680513:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067282688 00000100:00100000:0.0:1789762740.680520:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067283328 00000100:00100000:2.0:1789762740.680525:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067282688 00000100:00100000:0.0:1789762740.680527:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067283328 00000100:00100000:0.0:1789762740.680591:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 242 00000100:00100000:2.0:1789762740.680604:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 309 00000100:00100000:0.0:1789762740.680619:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067283328:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762740.680633:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1552450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067282688:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762740.680668:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067283328:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 52us (259us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762740.680680:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 242 00000100:00100000:2.0:1789762740.680690:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1552450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067282688:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 64us (1021us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762740.680701:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 309 00000100:00100000:0.0:1789762740.680713:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067283072 00000100:00100000:0.0:1789762740.680721:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067283072 00000100:00100000:0.0:1789762740.680785:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 96 00000100:00100000:0.0:1789762740.680796:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdddf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067283072:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762740.680854:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdddf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067283072:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 59us (879us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762740.680863:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 96 00000001:00080000:0.0:1789762740.683769:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967531 is committed 00000100:00100000:0.0:1789762740.686528:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067283456 00000100:00100000:0.0:1789762740.686534:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067283456 00000100:00100000:0.0:1789762740.686564:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 213914933439 time 213915071084 state 0x0 ref 1 00000100:00100000:0.0:1789762740.686582:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214115071084 time 213915071084 state 0x1 ref 1 00010000:00010000:2.0:1789762740.864077:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762740.868126:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214115071084 time 214115198973 state 0x0 ref 1 00000100:00100000:2.0:1789762740.868164:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067283456:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762740.868237:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 80740352, mtime: 0x6aad9cb4, atime: 0x6aad9ca5, ctime: 0x6aad9cb4, blocks: 0x2680d 00010000:00010000:2.0:1789762740.868249:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174862 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762740.868272:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=214s v=(0 0) 00010000:00010000:2.0:1789762740.868278:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762740.868318:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174862 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762740.868339:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000055ec1a24/0xdf5a7ac59f174862 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762740.880779:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967532 is committed 00000100:00100000:0.0:1789762740.883392:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067283584 00000100:00100000:0.0:1789762740.883402:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067283584 00000100:00100000:0.0:1789762740.883457:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214115071084 time 214115198973 state 0x0 ref 1 00000100:00100000:0.0:1789762740.883469:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214315198973 time 214115198973 state 0x1 ref 1 00000100:00100000:2.0:1789762741.068260:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214315198973 time 214315328971 state 0x0 ref 1 00000100:00100000:2.0:1789762741.068322:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067283584:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762741.068402:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 81788928, mtime: 0x6aad9cb4, atime: 0x6aad9ca5, ctime: 0x6aad9cb4, blocks: 0x2700d 00010000:00010000:2.0:1789762741.068412:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174869 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762741.068429:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=214s v=(0 0) 00010000:00010000:2.0:1789762741.068434:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762741.068439:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174869 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762741.068454:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174869 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762741.081048:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 81788928 -> 82837504 00002000:00010000:2.0:1789762741.081056:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 159757 -> 161805 00010000:00010000:2.0:1789762741.081074:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174869 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762741.081087:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174869 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762741.081104:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174869 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762741.081110:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee8cf000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762741.081115:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174869 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762741.081123:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174869 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762741.081143:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174869 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762741.081150:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd6b1330/0xdf5a7ac59f174869 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762741.081252:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067283584:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12969us (197980us total) trans 4294967533 rc 0/0 00000100:00100000:2.0:1789762741.081262:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 113 00000100:00100000:2.0:1789762741.081271:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214315198973 time 214315328971 state 0x2 ref 0 00000100:00100000:3.0:1789762741.083763:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067283712 00000100:00100000:3.0:1789762741.083773:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067283712 00000100:00100000:3.0:1789762741.083820:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214315198973 time 214315328971 state 0x0 ref 1 00000100:00100000:3.0:1789762741.083831:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214515328971 time 214315328971 state 0x1 ref 1 00000001:00080000:2.0:1789762741.280540:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967534 is committed 00000100:00100000:3.0:1789762741.283173:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067283840 00000100:00100000:3.0:1789762741.283183:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067283840 00000100:00100000:3.0:1789762741.283234:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214515328971 time 214515448341 state 0x0 ref 1 00000100:00100000:3.0:1789762741.283248:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214715448341 time 214515448341 state 0x1 ref 1 00000001:00080000:2.0:1789762741.480088:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967535 is committed 00000100:00100000:2.0:1789762741.482704:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762741.482788:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067283968 00000100:00100000:0.0:1789762741.482799:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067283968 00000100:00100000:0.0:1789762741.482844:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214715448341 time 214715553603 state 0x0 ref 1 00000100:00100000:0.0:1789762741.482855:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214915553603 time 214715553603 state 0x1 ref 1 00000100:00100000:0.0:1789762741.668627:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214915553603 time 214915691250 state 0x0 ref 1 00000100:00100000:0.0:1789762741.668671:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067283968:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762741.668744:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 84934656, mtime: 0x6aad9cb5, atime: 0x6aad9ca5, ctime: 0x6aad9cb5, blocks: 0x2880d 00010000:00010000:0.0:1789762741.668753:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f17487e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.668773:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=214s v=(0 0) 00010000:00010000:0.0:1789762741.668778:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762741.668783:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f17487e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.668797:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f17487e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762741.682260:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967536 is committed 00002000:00010000:3.0:1789762741.682264:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 84934656 -> 85983232 00002000:00010000:3.0:1789762741.682272:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 165901 -> 167949 00010000:00010000:3.0:1789762741.682305:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f17487e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.682316:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f17487e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.682325:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f17487e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.682333:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762741.682338:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f17487e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.682348:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f17487e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.682370:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f17487e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.682378:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f17487e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762741.682457:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067283968:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13805us (199750us total) trans 4294967536 rc 0/0 00000100:00100000:3.0:1789762741.682468:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 116 00000100:00100000:3.0:1789762741.682475:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 214915553603 time 214915691250 state 0x2 ref 0 00000100:00100000:0.0:1789762741.685466:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762741.760328:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074623104:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762741.760334:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6095500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074623232:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.760431:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762741.760440:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762741.760489:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee70dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074623744:0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.760496:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6095c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074622976:0@lo:400:kworker.0 00000100:00100000:3.0:1789762741.760499:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d98af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074623360:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.760560:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762741.760567:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762741.760627:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762741.760639:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074623616:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762741.760642:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee70c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074624256:0@lo:400:kworker.0 00000100:00100000:3.0:1789762741.760678:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee70ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074623872:0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.760689:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762741.760719:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762741.760723:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee70d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074624128:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.760779:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff880000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074624768:0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.760789:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074622976 00000100:00100000:3.0:1789762741.760789:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762741.760798:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074622976 00000100:00100000:0.0:1789762741.760845:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762741.760859:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee70ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074624384:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.760878:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789762741.760879:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 310 00000100:00100000:0.0:1789762741.760888:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff882680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074624640:0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.760900:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb939450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074622976:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.760958:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762741.760972:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762741.761027:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074622848 00000100:00100000:3.0:1789762741.761030:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff883b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074624896:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.761033:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074622848 00000100:00100000:2.0:1789762741.761033:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb939450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074622976:12345-0@lo:400:kworker.0 Request processed in 140us (480us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762741.761057:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 310 00000100:00100000:0.0:1789762741.761089:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074623104 00000100:00100000:0.0:1789762741.761095:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074623104 00000100:00100000:3.0:1789762741.761116:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762741.761121:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074624000 00000100:00100000:2.0:1789762741.761127:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074624000 00000100:00100000:0.0:1789762741.761144:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074623616 00000100:00100000:0.0:1789762741.761155:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074623616 00000100:00100000:2.0:1789762741.761176:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 102 00000100:00100000:2.0:1789762741.761189:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074622848:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762741.761202:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 103 00000100:00100000:3.0:1789762741.761210:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 244 00002000:00000024:2.0:1789762741.761215:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 211091456 pending 0 free 3779067904 avail 3771727872 00000100:00100000:0.0:1789762741.761220:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3befb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074623104:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762741.761242:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074623488:12345-0@lo:400:kworker.0 00002000:00000024:0.0:1789762741.761248:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 211091456 pending 0 free 3779067904 avail 3771727872 00000100:00100000:2.0:1789762741.761298:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074622848:12345-0@lo:13:osp-pre-0-1.0 Request processed in 94us (943us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762741.761312:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 102 00000100:00100000:0.0:1789762741.761341:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3befb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074623104:12345-0@lo:13:osp-pre-0-0.0 Request processed in 126us (905us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762741.761360:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 104 00000100:00100000:3.0:1789762741.761364:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074623488:12345-0@lo:400:kworker.0 Request processed in 128us (850us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762741.761370:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 103 00000100:00100000:2.0:1789762741.761372:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074624000:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762741.761393:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 244 00002000:00000024:2.0:1789762741.761417:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762741.761419:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 105 00000100:00100000:3.0:1789762741.761448:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:0.0:1789762741.761449:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074623616:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789762741.761473:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762741.761473:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074623872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.761474:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074624000:12345-0@lo:13:osp-pre-1-0.0 Request processed in 103us (826us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762741.761507:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 104 00000100:00100000:0.0:1789762741.761532:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074623616:12345-0@lo:13:osp-pre-1-1.0 Request processed in 94us (844us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762741.761540:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 105 00000100:00100000:2.0:1789762741.761555:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074624512 00000100:00100000:2.0:1789762741.761560:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074624512 00000100:00100000:0.0:1789762741.761597:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074623744 00000100:00100000:0.0:1789762741.761607:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074623744 00000100:00100000:3.0:1789762741.761608:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074623872:12345-0@lo:400:kworker.0 Request processed in 135us (823us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762741.761613:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074624768 00000100:00100000:2.0:1789762741.761623:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074624768 00000100:00100000:3.0:1789762741.761638:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00100000:2.0:1789762741.761652:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074624640 00000100:00100000:2.0:1789762741.761657:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074624640 00000100:00100000:0.0:1789762741.761672:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074623360 00000100:00100000:0.0:1789762741.761681:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074623360 00000100:00100000:2.0:1789762741.761687:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 97 00000100:00100000:2.0:1789762741.761709:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074624512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762741.761725:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762741.761731:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:0.0:1789762741.761745:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdddf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074623744:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.761792:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074624512:12345-0@lo:400:kworker.0 Request processed in 96us (1025us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762741.761805:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 97 00000100:00100000:0.0:1789762741.761843:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdddf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074623744:12345-0@lo:400:kworker.0 Request processed in 100us (1290us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762741.761854:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bee300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074623104:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762741.761860:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00100000:2.0:1789762741.761869:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 99 00000100:00100000:2.0:1789762741.761892:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdddce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074624640:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762741.761902:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6095c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074622976:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.761903:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 171 00000100:00100000:0.0:1789762741.761964:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff883480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074623360:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762741.761995:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074623616:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762741.762063:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdddce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074624640:12345-0@lo:400:kworker.0 Request processed in 172us (1101us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762741.762064:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff883480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074623360:12345-0@lo:400:kworker.0 Request processed in 137us (1435us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762741.762107:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 171 00000100:00100000:2.0:1789762741.762109:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 99 00000100:00100000:3.0:1789762741.762116:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074623488:0@lo:400:kworker.0 00000100:00100000:3.0:1789762741.762166:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074624000:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762741.762218:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff882300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074624512:0@lo:400:kworker.0 00000100:00080000:0.0:1789762741.762254:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762741.762262:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d179c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074622848:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1789762741.762266:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762741.762276:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762741.762292:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762741.762302:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff882680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074624640:0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.762351:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff880000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074624768:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.762354:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d98af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074623360:0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.762384:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6095500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074623232:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.762399:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074623872:0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.762413:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074623744:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.762424:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074624384:0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.762430:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074624256:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.762484:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff883b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074624896:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.868717:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 215115691250 time 215115784995 state 0x0 ref 1 00000100:00100000:0.0:1789762741.868753:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdddd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067284096:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762741.868825:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 85983232, mtime: 0x6aad9cb5, atime: 0x6aad9ca5, ctime: 0x6aad9cb5, blocks: 0x2900d 00010000:00010000:0.0:1789762741.868835:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174885 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.868852:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=215s v=(0 0) 00010000:00010000:0.0:1789762741.868856:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762741.868862:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174885 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.868878:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174885 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762741.884838:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967537 is committed 00000100:00100000:0.0:1789762741.887291:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:0.0:1789762741.888034:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762742.068793:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 215315784995 time 215315862990 state 0x0 ref 1 00000100:00100000:3.0:1789762742.068823:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdddd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067284224:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762742.068888:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 87031808, mtime: 0x6aad9cb5, atime: 0x6aad9ca5, ctime: 0x6aad9cb5, blocks: 0x2980d 00010000:00010000:3.0:1789762742.068898:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f17488c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.068914:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=215s v=(0 0) 00010000:00010000:3.0:1789762742.068945:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762742.068952:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f17488c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.068967:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f17488c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762742.080537:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967538 is committed 00000100:00100000:2.0:1789762742.082842:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762742.082882:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067284352 00000100:00100000:3.0:1789762742.082891:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067284352 00000100:00100000:3.0:1789762742.082952:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 215315784995 time 215315862990 state 0x0 ref 1 00000100:00100000:3.0:1789762742.082964:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 215515862990 time 215315862990 state 0x1 ref 1 00000100:00100000:3.0:1789762742.268830:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 215515862990 time 215515900968 state 0x0 ref 1 00000100:00100000:3.0:1789762742.268855:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067284352:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762742.268913:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 88080384, mtime: 0x6aad9cb5, atime: 0x6aad9ca5, ctime: 0x6aad9cb5, blocks: 0x2a00d 00010000:00010000:3.0:1789762742.268941:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174893 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.268953:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=215s v=(0 0) 00010000:00010000:3.0:1789762742.268957:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762742.268961:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174893 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.268973:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174893 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762742.282033:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967539 is committed 00002000:00010000:3.0:1789762742.282040:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 88080384 -> 89128960 00002000:00010000:3.0:1789762742.282046:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 172045 -> 174093 00010000:00010000:3.0:1789762742.282058:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174893 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.282067:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174893 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.282073:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174893 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.282078:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762742.282081:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174893 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.282098:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174893 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.282114:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174893 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.282119:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f174893 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762742.282180:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067284352:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13334us (199336us total) trans 4294967539 rc 0/0 00000100:00100000:3.0:1789762742.282188:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 119 00000100:00100000:3.0:1789762742.282194:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 215515862990 time 215515900968 state 0x2 ref 0 00000100:00100000:0.0:1789762742.284829:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762742.468964:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 215715900968 time 215716033818 state 0x0 ref 1 00000100:00100000:0.0:1789762742.469000:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdddd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067284480:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762742.469062:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 89128960, mtime: 0x6aad9cb5, atime: 0x6aad9ca5, ctime: 0x6aad9cb5, blocks: 0x2a80d 00010000:00010000:0.0:1789762742.469070:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17489a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.469103:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=215s v=(0 0) 00010000:00010000:0.0:1789762742.469107:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762742.469111:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17489a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.469123:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17489a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762742.481397:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 89128960 -> 90177536 00000001:00080000:3.0:1789762742.481402:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967540 is committed 00002000:00010000:2.0:1789762742.481403:0:13300:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb mtime from disk: 1789762741 -> 1789762742 00002000:00010000:2.0:1789762742.481409:0:13300:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb ctime from disk: 1789762741 -> 1789762742 00002000:00010000:2.0:1789762742.481411:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 174093 -> 176141 00010000:00010000:2.0:1789762742.481421:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17489a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762742.481431:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17489a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762742.481438:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17489a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762742.481443:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762742.481447:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17489a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762742.481453:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17489a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762742.481467:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17489a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762742.481472:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f17489a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762742.481540:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdddd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067284480:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12557us (196712us total) trans 4294967540 rc 0/0 00000100:00100000:2.0:1789762742.481547:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 120 00000100:00100000:2.0:1789762742.481553:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 215715900968 time 215716033818 state 0x2 ref 0 00000100:00100000:0.0:1789762742.483710:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762742.669083:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 215916033818 time 215916154108 state 0x0 ref 1 00000100:00100000:3.0:1789762742.669117:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067284608:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762742.669175:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 90177536, mtime: 0x6aad9cb6, atime: 0x6aad9ca5, ctime: 0x6aad9cb6, blocks: 0x2b00d 00010000:00010000:3.0:1789762742.669183:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.669200:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=215s v=(0 0) 00010000:00010000:3.0:1789762742.669234:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762742.669242:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.669296:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762742.684141:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 90177536 -> 91226112 00000001:00080000:2.0:1789762742.684148:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967541 is committed 00002000:00010000:3.0:1789762742.684148:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 176141 -> 178189 00010000:00010000:3.0:1789762742.684162:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.684174:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.684181:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.684186:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762742.684190:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.684198:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.684213:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.684218:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762742.684301:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067284608:12345-192.168.203.37@tcp:4:dd.500 Request processed in 15199us (200589us total) trans 4294967541 rc 0/0 00000100:00100000:3.0:1789762742.684309:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 121 00000100:00100000:3.0:1789762742.684316:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 215916033818 time 215916154108 state 0x2 ref 0 00000100:00100000:3.0:1789762742.686798:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067284736 00000100:00100000:3.0:1789762742.686811:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067284736 00000100:00100000:3.0:1789762742.686860:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 215916033818 time 215916154108 state 0x0 ref 1 00000100:00100000:3.0:1789762742.686869:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216116154108 time 215916154108 state 0x1 ref 1 00000100:00100000:0.0:1789762742.869200:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216116154108 time 216116272986 state 0x0 ref 1 00000100:00100000:0.0:1789762742.869228:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067284736:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762742.869296:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 91226112, mtime: 0x6aad9cb6, atime: 0x6aad9ca5, ctime: 0x6aad9cb6, blocks: 0x2b80d 00010000:00010000:0.0:1789762742.869303:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f1748a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.869317:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=216s v=(0 0) 00010000:00010000:0.0:1789762742.869320:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762742.869325:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f1748a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.869338:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f1748a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762742.880550:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967542 is committed 00000100:00100000:2.0:1789762742.882970:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762742.883019:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067284864 00000100:00100000:0.0:1789762742.883032:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067284864 00000100:00100000:0.0:1789762742.883066:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216116154108 time 216116272986 state 0x0 ref 1 00000100:00100000:0.0:1789762742.883075:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216316272986 time 216116272986 state 0x1 ref 1 00000100:00100000:0.0:1789762743.069265:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216316272986 time 216316335404 state 0x0 ref 1 00000100:00100000:0.0:1789762743.069341:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3beca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067284864:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762743.069404:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 92274688, mtime: 0x6aad9cb6, atime: 0x6aad9ca5, ctime: 0x6aad9cb6, blocks: 0x2c00d 00010000:00010000:0.0:1789762743.069414:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f1748af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.069429:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=216s v=(0 0) 00010000:00010000:0.0:1789762743.069433:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.069438:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f1748af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.069451:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f1748af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762743.082447:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967543 is committed 00002000:00010000:0.0:1789762743.082466:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 92274688 -> 93323264 00002000:00010000:0.0:1789762743.082472:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 180237 -> 182285 00010000:00010000:0.0:1789762743.082483:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f1748af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.082493:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f1748af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.082512:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f1748af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.082518:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762743.082521:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f1748af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.082546:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f1748af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.082571:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f1748af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.082582:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f1748af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762743.082686:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3beca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067284864:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13385us (199712us total) trans 4294967543 rc 0/0 00000100:00100000:0.0:1789762743.082701:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 123 00000100:00100000:0.0:1789762743.082711:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216316272986 time 216316335404 state 0x2 ref 0 00000100:00100000:2.0:1789762743.085357:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762743.085403:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067284992 00000100:00100000:3.0:1789762743.085411:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067284992 00000100:00100000:3.0:1789762743.085447:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216316272986 time 216316335404 state 0x0 ref 1 00000100:00100000:3.0:1789762743.085456:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216516335404 time 216316335404 state 0x1 ref 1 00000001:00080000:2.0:1789762743.280515:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967544 is committed 00002000:00010000:0.0:1789762743.280521:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 93323264 -> 94371840 00002000:00010000:0.0:1789762743.280526:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 182285 -> 184333 00010000:00010000:0.0:1789762743.280537:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.280558:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.280564:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.280569:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762743.280572:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.280579:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.280594:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.280599:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762743.280662:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067284992:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11287us (195304us total) trans 4294967544 rc 0/0 00000100:00100000:0.0:1789762743.280670:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 124 00000100:00100000:0.0:1789762743.280677:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216516335404 time 216516430615 state 0x2 ref 0 00000100:00100000:0.0:1789762743.282978:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:0.0:1789762743.483044:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967545 is committed 00000100:00100000:2.0:1789762743.486054:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762743.486095:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067285248 00000100:00100000:0.0:1789762743.486103:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067285248 00000100:00100000:0.0:1789762743.486136:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216716430615 time 216716510571 state 0x0 ref 1 00000100:00100000:0.0:1789762743.486145:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216916510571 time 216716510571 state 0x1 ref 1 00000100:00100000:0.0:1789762743.669499:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216916510571 time 216916571638 state 0x0 ref 1 00000100:00100000:0.0:1789762743.669528:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067285248:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762743.669584:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 95420416, mtime: 0x6aad9cb7, atime: 0x6aad9ca5, ctime: 0x6aad9cb7, blocks: 0x2d80d 00010000:00010000:0.0:1789762743.669594:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000052f30ec7/0xdf5a7ac59f1748c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.669611:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=216s v=(0 0) 00010000:00010000:0.0:1789762743.669617:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.669624:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052f30ec7/0xdf5a7ac59f1748c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.669636:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052f30ec7/0xdf5a7ac59f1748c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762743.687781:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967546 is committed 00000100:00100000:2.0:1789762743.691668:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762743.691764:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067285376 00000100:00100000:0.0:1789762743.691779:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067285376 00000100:00100000:0.0:1789762743.691853:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 216916510571 time 216916571638 state 0x0 ref 1 00000100:00100000:0.0:1789762743.691866:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 217116571638 time 216916571638 state 0x1 ref 1 00000100:00100000:3.0:1789762743.808094:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074625024:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762743.808161:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762743.808205:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074625024 00000100:00100000:0.0:1789762743.808213:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074625024 00000100:00100000:0.0:1789762743.808273:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:0.0:1789762743.808287:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074625024:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762743.808323:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762743.808371:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074625024:12345-0@lo:41:osp-pre-1-0.0 Request processed in 88us (209us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762743.808386:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00100000:3.0:1789762743.808419:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bee680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074625024:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762743.869559:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 217116571638 time 217116631125 state 0x0 ref 1 00000100:00100000:0.0:1789762743.869586:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067285376:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762743.869645:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 96468992, mtime: 0x6aad9cb7, atime: 0x6aad9ca5, ctime: 0x6aad9cb7, blocks: 0x2e00d 00010000:00010000:0.0:1789762743.869653:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f1748cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.869666:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=217s v=(0 0) 00010000:00010000:0.0:1789762743.869669:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.869674:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f1748cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.869686:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f1748cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762743.881088:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967547 is committed 00000100:00100000:0.0:1789762743.883582:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:0.0:1789762743.936030:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762744.069623:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 217316631125 time 217316695375 state 0x0 ref 1 00000100:00100000:3.0:1789762744.069646:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdddc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067285504:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762744.069692:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 97517568, mtime: 0x6aad9cb7, atime: 0x6aad9ca5, ctime: 0x6aad9cb7, blocks: 0x2e80d 00010000:00010000:3.0:1789762744.069700:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.069730:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=217s v=(0 0) 00010000:00010000:3.0:1789762744.069733:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762744.069738:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.069749:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762744.082287:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 97517568 -> 98566144 00000001:00080000:2.0:1789762744.082295:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967548 is committed 00002000:00010000:0.0:1789762744.082327:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 190477 -> 192525 00010000:00010000:0.0:1789762744.082341:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.082355:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.082367:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.082375:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762744.082399:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.082415:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.082441:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.082471:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1748d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762744.082577:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdddc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067285504:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12933us (198990us total) trans 4294967548 rc 0/0 00000100:00100000:0.0:1789762744.082592:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 128 00000100:00100000:0.0:1789762744.082601:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 217316631125 time 217316695375 state 0x2 ref 0 00000100:00100000:2.0:1789762744.085834:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000001:00080000:2.0:1789762744.280781:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967549 is committed 00002000:00010000:0.0:1789762744.280833:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 98566144 -> 99614720 00002000:00010000:0.0:1789762744.280838:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 192525 -> 194573 00010000:00010000:0.0:1789762744.280848:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.280856:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.280862:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.280867:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762744.280870:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.280876:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.280889:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.280894:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f1748d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762744.280981:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067285632:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11259us (195147us total) trans 4294967549 rc 0/0 00000100:00100000:0.0:1789762744.280998:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 129 00000100:00100000:0.0:1789762744.281003:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 217516695375 time 217516776398 state 0x2 ref 0 00000100:00100000:0.0:1789762744.283286:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762744.469835:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 217716776398 time 217716906688 state 0x0 ref 1 00000100:00100000:0.0:1789762744.469863:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdddf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067285760:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762744.469914:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 99614720, mtime: 0x6aad9cb7, atime: 0x6aad9ca5, ctime: 0x6aad9cb7, blocks: 0x2f80d 00010000:00010000:0.0:1789762744.469945:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1748e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.469958:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=217s v=(0 0) 00010000:00010000:0.0:1789762744.469961:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762744.469966:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1748e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.469978:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1748e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762744.481370:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967550 is committed 00002000:00010000:3.0:1789762744.481377:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 99614720 -> 100663296 00002000:00010000:3.0:1789762744.481382:0:14171:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb mtime from disk: 1789762743 -> 1789762744 00002000:00010000:3.0:1789762744.481384:0:14171:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb ctime from disk: 1789762743 -> 1789762744 00002000:00010000:3.0:1789762744.481386:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 194573 -> 196621 00010000:00010000:3.0:1789762744.481395:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1748e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.481404:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1748e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.481411:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1748e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.481416:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762744.481420:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1748e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.481437:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1748e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.481452:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1748e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.481457:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000684da15/0xdf5a7ac59f1748e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762744.481514:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdddf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067285760:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11663us (198227us total) trans 4294967550 rc 0/0 00000100:00100000:3.0:1789762744.481522:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 130 00000100:00100000:3.0:1789762744.481528:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 217716776398 time 217716906688 state 0x2 ref 0 00000100:00100000:0.0:1789762744.483907:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067285888 00000100:00100000:0.0:1789762744.483915:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067285888 00000100:00100000:0.0:1789762744.483961:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 217716776398 time 217716906688 state 0x0 ref 1 00000100:00100000:0.0:1789762744.483971:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 217916906688 time 217716906688 state 0x1 ref 1 00000100:00100000:0.0:1789762744.669884:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 217916906688 time 217916956351 state 0x0 ref 1 00000100:00100000:0.0:1789762744.669949:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067285888:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762744.669998:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 100663296, mtime: 0x6aad9cb8, atime: 0x6aad9ca5, ctime: 0x6aad9cb8, blocks: 0x3000d 00010000:00010000:0.0:1789762744.670006:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f1748e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.670019:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=217s v=(0 0) 00010000:00010000:0.0:1789762744.670023:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762744.670027:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f1748e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.670039:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f1748e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762744.681146:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967551 is committed 00002000:00010000:0.0:1789762744.681196:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 100663296 -> 101711872 00002000:00010000:0.0:1789762744.681200:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 196621 -> 198669 00010000:00010000:0.0:1789762744.681211:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f1748e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.681220:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f1748e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.681226:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f1748e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.681231:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762744.681235:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f1748e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.681249:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f1748e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.681279:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f1748e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.681285:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f1748e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762744.681344:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067285888:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11426us (197477us total) trans 4294967551 rc 0/0 00000100:00100000:0.0:1789762744.681353:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 131 00000100:00100000:0.0:1789762744.681359:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 217916906688 time 217916956351 state 0x2 ref 0 00000100:00100000:2.0:1789762744.683543:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762744.683578:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067286016 00000100:00100000:0.0:1789762744.683587:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067286016 00000100:00100000:0.0:1789762744.683626:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 217916906688 time 217916956351 state 0x0 ref 1 00000100:00100000:0.0:1789762744.683637:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218116956351 time 217916956351 state 0x1 ref 1 00000100:00100000:0.0:1789762744.832419:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074625152 00000100:00100000:0.0:1789762744.832431:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074625152 00000100:00100000:0.0:1789762744.832502:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:0.0:1789762744.832515:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074625152:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762744.832569:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762744.832645:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074625152:12345-0@lo:41:osp-pre-0-1.0 Request processed in 134us (298us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762744.832659:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000100:00100000:3.0:1789762744.869989:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218116956351 time 218117055400 state 0x0 ref 1 00000100:00100000:3.0:1789762744.870048:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067286016:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762744.870135:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 101711872, mtime: 0x6aad9cb8, atime: 0x6aad9ca5, ctime: 0x6aad9cb8, blocks: 0x3080d 00010000:00010000:3.0:1789762744.870143:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1748ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.870156:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=218s v=(0 0) 00010000:00010000:3.0:1789762744.870160:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762744.870165:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1748ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.870192:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1748ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762744.884288:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967552 is committed 00002000:00010000:3.0:1789762744.884297:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 101711872 -> 102760448 00002000:00010000:3.0:1789762744.884303:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 198669 -> 200717 00010000:00010000:3.0:1789762744.884315:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1748ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.884328:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1748ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.884339:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1748ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.884348:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762744.884354:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1748ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.884366:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1748ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.884389:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1748ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.884400:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1748ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762744.884483:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067286016:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14454us (200938us total) trans 4294967552 rc 0/0 00000100:00100000:3.0:1789762744.884492:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 132 00000100:00100000:3.0:1789762744.884499:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218116956351 time 218117055400 state 0x2 ref 0 00000100:00100000:3.0:1789762744.887388:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067286144 00000100:00100000:3.0:1789762744.887398:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067286144 00000100:00100000:3.0:1789762744.887438:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218116956351 time 218117055400 state 0x0 ref 1 00000100:00100000:3.0:1789762744.887451:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218317055400 time 218117055400 state 0x1 ref 1 00010000:00010000:2.0:1789762744.960050:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762745.070226:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789762745.090209:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967553 is committed 00002000:00010000:0.0:1789762745.090335:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb size from disk: 102760448 -> 103809024 00002000:00010000:0.0:1789762745.090342:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x4:0x0] updating lvb blocks from disk: 200717 -> 202765 00010000:00010000:0.0:1789762745.090355:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f1748f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.090367:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f1748f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.090376:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f1748f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.090382:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762745.090387:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f1748f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.090395:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f1748f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.090435:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f1748f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.090445:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f1748f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762745.090570:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067286144:12345-192.168.203.37@tcp:4:dd.500 Request processed in 20436us (203221us total) trans 4294967553 rc 0/0 00000100:00100000:0.0:1789762745.090586:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 133 00000100:00100000:0.0:1789762745.090597:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218317055400 time 218317174155 state 0x2 ref 0 00000100:00100000:2.0:1789762745.094600:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762745.094730:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067286272 00000100:00100000:0.0:1789762745.094745:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067286272 00000100:00100000:0.0:1789762745.094811:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218317055400 time 218317174155 state 0x0 ref 1 00000100:00100000:0.0:1789762745.094823:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218517174155 time 218317174155 state 0x1 ref 1 00000001:00080000:0.0:1789762745.288513:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967554 is committed 00000100:00100000:2.0:1789762745.291402:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00080000:0.0:1789762745.291567:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762745.364845:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762745.364899:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067286784 00000100:00100000:3.0:1789762745.364910:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067286784 00000100:00100000:3.0:1789762745.364987:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 154 00000100:00100000:3.0:1789762745.364996:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:12196:x1876702067286784:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762745.365022:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762745.365039:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f174903 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c51b expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365102:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f17490a lrc: 3/1,0 mode: --/CR res: [0x200000402:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365113:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f17490a lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365121:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f17490a lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365147:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174911 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365153:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174911 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365157:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174911 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365619:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174918 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365630:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174918 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365637:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174918 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762745.365647:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb:0x0]: returning size 104857600 00010000:00010000:3.0:1789762745.365734:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f17490a lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365739:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f17490a lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365745:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f17490a lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365749:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762745.365752:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f17490a lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365766:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f17490a lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365777:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:3.0:1789762745.365779:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762745.365782:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f17490a lrc: 2/0,0 mode: --/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365786:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f17490a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365796:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174911 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365800:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174911 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365804:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174911 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365807:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762745.365809:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174911 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365813:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174911 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365817:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x4009f790 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762745.365818:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762745.365820:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174911 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365830:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174911 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365862:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f174903 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c51b expref: 13 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365871:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174918 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c51b expref: 12 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.365876:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37db743c00) 00010000:00010000:3.0:1789762745.365878:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762745.365880:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff935a00, rc 0) 00000100:00100000:3.0:1789762745.365966:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:12196:x1876702067286784:12345-192.168.203.37@tcp:101:dd.500 Request processed in 970us (1129us total) trans 4294967357 rc 0/0 00000100:00100000:3.0:1789762745.365972:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 154 00000100:00100000:2.0:1789762745.367481:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762745.370028:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762745.370062:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067287040 00000100:00100000:3.0:1789762745.370067:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067287040 00000100:00100000:3.0:1789762745.370093:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218517174155 time 218517418991 state 0x0 ref 1 00000100:00100000:3.0:1789762745.370102:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218717418991 time 218517418991 state 0x1 ref 1 00000100:00100000:0.0:1789762745.470435:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218717418991 time 218717507443 state 0x0 ref 1 00000100:00100000:0.0:1789762745.470464:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067287040:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762745.470501:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f17492d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.470514:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762745.470518:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762745.470523:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f17492d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.470534:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f17492d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.474648:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f17492d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.474658:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f17492d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.474664:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f17492d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.474670:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762745.474673:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f17492d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.474679:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f17492d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.474693:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f17492d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.474698:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a21d7aff/0xdf5a7ac59f17492d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762745.474747:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067287040:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4296us (104721us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762745.474755:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 135 00000100:00100000:0.0:1789762745.474762:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218717418991 time 218717507443 state 0x2 ref 0 00000100:00100000:2.0:1789762745.476749:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762745.476796:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067287168 00000100:00100000:0.0:1789762745.476805:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067287168 00000100:00100000:0.0:1789762745.476839:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218717418991 time 218717507443 state 0x0 ref 1 00000100:00100000:0.0:1789762745.476851:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 218917507443 time 218717507443 state 0x1 ref 1 00000100:00100000:2.0:1789762745.675681:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762745.800027:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762745.800125:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762745.800134:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067287424 00000100:00100000:2.0:1789762745.800155:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067287424 00000100:00100000:3.0:1789762745.800224:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067287552 00000100:00100000:3.0:1789762745.800234:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067287552 00000100:00100000:2.0:1789762745.800235:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 311 00000100:00100000:2.0:1789762745.800272:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb938850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067287424:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:0.0:1789762745.800306:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762745.800338:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 249 00000100:00100000:3.0:1789762745.800350:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdddf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067287552:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762745.800376:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb938850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067287424:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 123us (357us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762745.800391:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 311 00000100:00100000:3.0:1789762745.800416:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdddf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067287552:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 68us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762745.800440:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 249 00000100:00100000:2.0:1789762745.802175:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762745.802212:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762745.802236:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762745.802276:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067287680 00000100:00100000:0.0:1789762745.802282:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067287680 00000100:00100000:3.0:1789762745.802304:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067287808 00000100:00100000:3.0:1789762745.802312:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067287808 00000100:00100000:0.0:1789762745.802349:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 100 00000100:00100000:0.0:1789762745.802363:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2394:x1876702067287680:12345-192.168.203.37@tcp:9:kworker.0 00000100:00080000:2.0:1789762745.802368:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762745.802370:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 251 00000100:00080000:2.0:1789762745.802374:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762745.802382:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762745.802383:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067287808:12345-192.168.203.37@tcp:400:kworker.0 00000020:00000080:0.0:1789762745.802393:0:8054:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xdf5a7ac59f1742ce 00000100:00080000:0.0:1789762745.802406:0:8054:0:(import.c:65:import_set_state_nolock()) ffff9c36cbb26000 : changing import state from FULL to CLOSED 00000100:00100000:3.0:1789762745.802425:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067287808:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 44us (184us total) trans 0 rc 0/0 00010000:00010000:2.0:1789762745.802430:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762745.802445:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 251 00000001:00000004:0.0:1789762745.802449:0:8054:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967345 00000100:00100000:0.0:1789762745.802704:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+2:2394:x1876702067287680:12345-192.168.203.37@tcp:9:kworker.0 Request processed in 347us (530us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762745.802710:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 100 00000001:00080000:3.0:1789762745.863598:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967356 is committed 00000001:00080000:3.0:1789762745.863623:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967357 is committed 00000100:00100000:2.0:1789762745.870622:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219117600978 time 219117694576 state 0x0 ref 1 00000100:00100000:2.0:1789762745.870650:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c8173800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067287296:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762745.870701:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17493b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.870719:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762745.870723:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762745.870730:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17493b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.870756:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17493b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.873605:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17493b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.873621:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17493b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.873631:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17493b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.873639:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762745.873643:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17493b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.873651:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17493b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.873679:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17493b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.873686:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17493b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762745.873738:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c8173800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067287296:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3100us (198059us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762745.873746:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 137 00000100:00100000:3.0:1789762745.873753:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219117600978 time 219117694576 state 0x2 ref 0 00000100:00100000:0.0:1789762745.875719:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762745.875757:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067288064 00000100:00100000:3.0:1789762745.875766:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067288064 00000100:00100000:3.0:1789762745.875802:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219117600978 time 219117694576 state 0x0 ref 1 00000100:00100000:3.0:1789762745.875812:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219317694576 time 219117694576 state 0x1 ref 1 00010000:00010000:2.0:1789762746.073688:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f174942 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762746.073699:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f174942 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762746.073706:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f174942 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762746.073711:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762746.073716:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f174942 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762746.073723:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f174942 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762746.073739:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f174942 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762746.073744:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f174942 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762746.073804:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdddea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067288064:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3065us (198086us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762746.073820:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 138 00000100:00100000:2.0:1789762746.073827:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219317694576 time 219317791316 state 0x2 ref 0 00000100:00100000:0.0:1789762746.075870:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762746.270826:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219517791316 time 219517892586 state 0x0 ref 1 00000100:00100000:3.0:1789762746.270872:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdddce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067288192:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762746.270953:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174949 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.270973:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762746.270977:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762746.270985:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174949 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.271022:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174949 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.275787:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174949 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.275800:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174949 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.275808:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174949 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.275815:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762746.275819:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174949 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.275826:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174949 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.275842:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174949 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.275847:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174949 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762746.275906:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdddce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067288192:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5058us (200038us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762746.275957:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 139 00000100:00100000:3.0:1789762746.275963:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219517791316 time 219517892586 state 0x2 ref 0 00000100:00100000:2.0:1789762746.277979:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762746.278015:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067288320 00000100:00100000:3.0:1789762746.278024:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067288320 00000100:00100000:3.0:1789762746.278062:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219517791316 time 219517892586 state 0x0 ref 1 00000100:00100000:3.0:1789762746.278072:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219717892586 time 219517892586 state 0x1 ref 1 00000100:00100000:0.0:1789762746.470949:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219717892586 time 219717992650 state 0x0 ref 1 00000100:00100000:0.0:1789762746.470989:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067288320:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762746.471058:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174950 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.471076:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762746.471109:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762746.471116:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174950 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.471132:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174950 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.475318:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174950 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.475332:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174950 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.475342:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174950 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.475351:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762746.475357:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174950 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.475369:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174950 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.475390:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174950 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.475399:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174950 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762746.475478:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067288320:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4507us (197500us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762746.475490:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 140 00000100:00100000:0.0:1789762746.475498:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219717892586 time 219717992650 state 0x2 ref 0 00000100:00100000:2.0:1789762746.478231:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067288448 00000100:00100000:2.0:1789762746.478266:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067288448 00000100:00100000:2.0:1789762746.478322:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219717892586 time 219717992650 state 0x0 ref 1 00000100:00100000:2.0:1789762746.478338:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219917992650 time 219717992650 state 0x1 ref 1 00000100:00100000:0.0:1789762746.671059:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219917992650 time 219918125951 state 0x0 ref 1 00000100:00100000:0.0:1789762746.671099:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067288448:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762746.671154:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174957 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.671186:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762746.671192:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762746.671200:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174957 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.671219:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174957 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.674235:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174957 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.674265:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174957 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.674277:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174957 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.674297:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762746.674303:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174957 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.674312:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174957 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.674331:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174957 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.674336:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f174957 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762746.674394:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067288448:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3316us (196224us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762746.674403:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 141 00000100:00100000:0.0:1789762746.674409:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219917992650 time 219918125951 state 0x2 ref 0 00000100:00100000:2.0:1789762746.676398:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762746.676441:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067288576 00000100:00100000:3.0:1789762746.676451:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067288576 00000100:00100000:3.0:1789762746.676493:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 219917992650 time 219918125951 state 0x0 ref 1 00000100:00100000:3.0:1789762746.676504:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220118125951 time 219918125951 state 0x1 ref 1 00000100:00100000:0.0:1789762746.871188:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220118125951 time 220118257796 state 0x0 ref 1 00000100:00100000:0.0:1789762746.871223:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067288576:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762746.871278:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17495e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.871292:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762746.871307:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762746.871312:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17495e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762746.871325:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17495e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.874610:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17495e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.874621:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17495e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.874629:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17495e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.874635:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762746.874639:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17495e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.874645:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17495e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.874671:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17495e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.874677:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17495e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762746.874725:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067288576:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3519us (198328us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762746.874731:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 142 00000100:00100000:3.0:1789762746.874736:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220118125951 time 220118257796 state 0x2 ref 0 00000100:00100000:2.0:1789762746.876865:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762746.876958:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067288704 00000100:00100000:0.0:1789762746.876966:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067288704 00000100:00100000:0.0:1789762746.876999:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220118125951 time 220118257796 state 0x0 ref 1 00000100:00100000:0.0:1789762746.877008:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220318257796 time 220118257796 state 0x1 ref 1 00000100:00100000:3.0:1789762746.880181:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074625536:0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.880217:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb931f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074625408:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762746.880265:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762746.880333:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762746.880378:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074626048:0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.880386:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee8c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074625920:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762746.880461:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762746.880473:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762746.880499:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074625408 00000100:00100000:0.0:1789762746.880507:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074625408 00000100:00100000:2.0:1789762746.880512:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074626432:0@lo:400:kworker.0 00000100:00100000:3.0:1789762746.880537:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e4ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074626560:0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.880563:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762746.880594:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762746.880607:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074625664 00000100:00100000:2.0:1789762746.880615:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074626944:0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.880617:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074625664 00000100:00100000:3.0:1789762746.880653:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8ab100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074627072:0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.880662:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074625920 00000100:00100000:0.0:1789762746.880667:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074625920 00000100:00100000:2.0:1789762746.880676:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762746.880718:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 106 00000100:00100000:3.0:1789762746.880720:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074626560 00000100:00100000:3.0:1789762746.880731:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074626560 00000100:00100000:0.0:1789762746.880738:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee8c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074625408:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762746.880756:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d179c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074625280:0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789762746.880764:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762746.880802:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762746.880810:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:3.0:1789762746.880826:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074626176:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.880847:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee8c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074625408:12345-0@lo:13:osp-pre-1-0.0 Request processed in 111us (520us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762746.880857:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074625792:0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.880858:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 106 00000100:00100000:3.0:1789762746.880878:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074626176:12345-0@lo:400:kworker.0 Request processed in 56us (354us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762746.880889:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074625280 00000100:00100000:0.0:1789762746.880895:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074625280 00000100:00100000:3.0:1789762746.880899:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00100000:3.0:1789762746.880974:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 254 00000100:00100000:0.0:1789762746.880986:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 107 00000100:00100000:2.0:1789762746.880998:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762746.881004:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074626432:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.881014:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074625664:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762746.881027:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074626304:0@lo:400:kworker.0 00002000:00000024:0.0:1789762746.881051:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762746.881071:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074626432:12345-0@lo:400:kworker.0 Request processed in 76us (511us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762746.881085:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 254 00000100:00100000:2.0:1789762746.881104:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762746.881119:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074625664:12345-0@lo:13:osp-pre-1-1.0 Request processed in 114us (714us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762746.881136:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762746.881147:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074626816:0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.881148:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 107 00000100:00100000:0.0:1789762746.881190:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 108 00000100:00100000:3.0:1789762746.881199:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e4ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074626560:0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.881209:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074625920:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762746.881226:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762746.881237:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074626944 00000100:00100000:3.0:1789762746.881258:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074626944 00002000:00000024:0.0:1789762746.881280:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762746.881301:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d19f4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074627328:0@lo:400:kworker.0 00000100:00100000:3.0:1789762746.881312:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074627072 00000100:00100000:3.0:1789762746.881317:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074627072 00000100:00100000:0.0:1789762746.881339:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074625920:12345-0@lo:13:osp-pre-0-0.0 Request processed in 134us (881us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762746.881350:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 108 00000100:00100000:2.0:1789762746.881366:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762746.881375:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 109 00000100:00100000:3.0:1789762746.881395:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074627328 00000100:00100000:0.0:1789762746.881401:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ebaa0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074625280:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762746.881405:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074627328 00002000:00000024:0.0:1789762746.881418:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3754950656 avail 3752853504 00000100:00100000:3.0:1789762746.881453:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 101 00000100:00100000:2.0:1789762746.881460:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074625792:0@lo:400:kworker.0 00000100:00100000:3.0:1789762746.881471:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074626944:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.881473:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074625280:12345-0@lo:13:osp-pre-0-1.0 Request processed in 87us (672us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762746.881487:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 109 00000100:00100000:2.0:1789762746.881524:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d179c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074625280:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762746.881553:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 102 00000100:00100000:3.0:1789762746.881561:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074626944:12345-0@lo:400:kworker.0 Request processed in 95us (888us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762746.881567:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074626816:0@lo:400:kworker.0 00000100:00100000:3.0:1789762746.881577:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 101 00000100:00100000:0.0:1789762746.881579:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074627072:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.881643:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074625536 00000100:00100000:2.0:1789762746.881649:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074625536 00000100:00100000:0.0:1789762746.881672:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074627072:12345-0@lo:400:kworker.0 Request processed in 106us (546us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762746.881684:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 102 00000100:00100000:3.0:1789762746.881689:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8ab100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074627072:0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.881730:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 312 00000100:00100000:2.0:1789762746.881747:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb93a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074625536:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.881752:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074626944:0@lo:400:kworker.0 00000100:00080000:3.0:1789762746.881771:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762746.881779:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789762746.881787:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789762746.881794:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762746.881801:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb931f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074625408:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762746.881831:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074626048 00000100:00100000:3.0:1789762746.881838:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074626048 00000100:00100000:0.0:1789762746.881839:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee8c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074625920:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762746.881839:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb93a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074625536:12345-0@lo:400:kworker.0 Request processed in 91us (1572us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762746.881866:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 312 00000100:00100000:3.0:1789762746.881878:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074626304 00000100:00100000:3.0:1789762746.881888:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074626304 00000100:00100000:0.0:1789762746.881897:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074626432:0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.881987:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:3.0:1789762746.882000:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:2.0:1789762746.882008:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074626048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762746.882027:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074626304:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.882072:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d19f4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074627328:0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.882095:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074626048:12345-0@lo:400:kworker.0 Request processed in 89us (1619us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762746.882097:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074626304:12345-0@lo:400:kworker.0 Request processed in 81us (997us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762746.882128:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00100000:2.0:1789762746.882137:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00100000:0.0:1789762746.882145:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074626304:0@lo:400:kworker.0 00000100:00100000:3.0:1789762746.882182:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074627200 00000100:00100000:3.0:1789762746.882186:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074627200 00000100:00080000:2.0:1789762746.882237:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762746.882237:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 104 00000100:00100000:0.0:1789762746.882269:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074625536:0@lo:400:kworker.0 00000100:00100000:3.0:1789762746.882273:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074627200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.882300:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074626048:0@lo:400:kworker.0 00000100:00100000:3.0:1789762746.882340:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074627200:12345-0@lo:400:kworker.0 Request processed in 70us (431us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762746.882347:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 104 00000100:00100000:0.0:1789762746.882376:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff861f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074626688:0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.882384:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d19f6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074627200:0@lo:400:kworker.0 00010000:00010000:0.0:1789762747.008120:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762747.071387:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762747.074478:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004a3f8694/0xdf5a7ac59f174965 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.074494:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a3f8694/0xdf5a7ac59f174965 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.074501:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a3f8694/0xdf5a7ac59f174965 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.074507:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7f42a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762747.074511:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a3f8694/0xdf5a7ac59f174965 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.074518:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a3f8694/0xdf5a7ac59f174965 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.074549:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a3f8694/0xdf5a7ac59f174965 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.074554:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a3f8694/0xdf5a7ac59f174965 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762747.074650:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067288704:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3315us (197792us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762747.074664:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 143 00000100:00100000:0.0:1789762747.074672:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220318257796 time 220318382123 state 0x2 ref 0 00000100:00100000:2.0:1789762747.076613:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762747.076663:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067288832 00000100:00100000:0.0:1789762747.076675:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067288832 00000100:00100000:0.0:1789762747.076718:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220318257796 time 220318382123 state 0x0 ref 1 00000100:00100000:0.0:1789762747.076733:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220518382123 time 220318382123 state 0x1 ref 1 00000100:00100000:2.0:1789762747.271430:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220518382123 time 220518503228 state 0x0 ref 1 00000100:00100000:2.0:1789762747.271471:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067288832:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762747.271510:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f17496c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762747.271525:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762747.271528:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762747.271533:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f17496c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762747.271544:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f17496c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.274578:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f17496c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.274600:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f17496c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.274613:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f17496c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.274620:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f1400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762747.274625:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f17496c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.274633:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f17496c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.274661:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f17496c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.274668:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f17496c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762747.274744:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067288832:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3278us (198127us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762747.274753:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 144 00000100:00100000:3.0:1789762747.274760:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220518382123 time 220518503228 state 0x2 ref 0 00000100:00100000:0.0:1789762747.276853:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762747.276887:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067288960 00000100:00100000:3.0:1789762747.276897:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067288960 00000100:00100000:3.0:1789762747.276954:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220518382123 time 220518503228 state 0x0 ref 1 00000100:00100000:3.0:1789762747.276965:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220718503228 time 220518503228 state 0x1 ref 1 00000001:00100000:3.0:1789762747.400039:0:9000:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000020:00000080:3.0:1789762747.400047:0:9000:0:(genops.c:885:class_export_put()) final put ffff9c36cbb24000/fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000100:00100000:2.0:1789762747.471551:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220718503228 time 220718617436 state 0x0 ref 1 00000100:00100000:2.0:1789762747.471592:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067288960:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762747.471651:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f174973 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762747.471675:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762747.471679:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762747.471686:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f174973 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762747.471701:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f174973 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762747.475442:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f174973 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762747.475454:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f174973 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762747.475462:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f174973 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762747.475468:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f1600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762747.475472:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f174973 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762747.475479:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f174973 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762747.475497:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f174973 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762747.475503:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f174973 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762747.475561:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067288960:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3992us (198709us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762747.475569:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 145 00000100:00100000:2.0:1789762747.475582:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220718503228 time 220718617436 state 0x2 ref 0 00000100:00100000:0.0:1789762747.477519:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762747.671702:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220918617436 time 220918765633 state 0x0 ref 1 00000100:00100000:3.0:1789762747.671747:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067289088:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762747.671807:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f17497a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.671829:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762747.671852:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762747.671861:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f17497a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.671878:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f17497a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.675660:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f17497a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.675682:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f17497a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.675693:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f17497a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.675703:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762747.675710:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f17497a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.675722:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f17497a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.675746:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f17497a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762747.675756:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f17497a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762747.675841:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067289088:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4113us (198319us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762747.675855:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 146 00000100:00100000:3.0:1789762747.675865:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220918617436 time 220918765633 state 0x2 ref 0 00000100:00100000:2.0:1789762747.677856:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762747.677897:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067289216 00000100:00100000:3.0:1789762747.677907:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067289216 00000100:00100000:3.0:1789762747.677998:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 220918617436 time 220918765633 state 0x0 ref 1 00000100:00100000:3.0:1789762747.678012:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221118765633 time 220918765633 state 0x1 ref 1 00000100:00100000:0.0:1789762747.871805:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221118765633 time 221118873426 state 0x0 ref 1 00000100:00100000:0.0:1789762747.871842:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067289216:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762747.871902:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174981 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.871950:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762747.871971:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762747.871978:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174981 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.871996:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174981 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.874829:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174981 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.874844:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174981 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.874854:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174981 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.874863:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1bf2c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762747.874868:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174981 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.874879:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174981 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.874899:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174981 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762747.874908:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174981 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762747.874993:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067289216:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3168us (197137us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762747.875003:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 147 00000100:00100000:0.0:1789762747.875011:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221118765633 time 221118873426 state 0x2 ref 0 00000100:00100000:2.0:1789762747.876962:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067289344 00000100:00100000:2.0:1789762747.876973:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067289344 00000100:00100000:2.0:1789762747.877020:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221118765633 time 221118873426 state 0x0 ref 1 00000100:00100000:2.0:1789762747.877031:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221318873426 time 221118873426 state 0x1 ref 1 00010000:00010000:2.0:1789762748.032026:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762748.071942:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221318873426 time 221318985580 state 0x0 ref 1 00000100:00100000:0.0:1789762748.071975:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067289344:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762748.072017:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174988 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.072034:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762748.072039:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762748.072045:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174988 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.072060:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174988 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.075032:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174988 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.075045:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174988 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.075068:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174988 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.075074:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1bf2c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762748.075077:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174988 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.075083:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174988 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.075096:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174988 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.075102:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f174988 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762748.075153:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067289344:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3192us (198244us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762748.075160:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 148 00000100:00100000:0.0:1789762748.075167:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221318873426 time 221318985580 state 0x2 ref 0 00000100:00100000:2.0:1789762748.077033:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762748.077082:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067289472 00000100:00100000:3.0:1789762748.077094:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067289472 00000100:00100000:3.0:1789762748.077146:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221318873426 time 221318985580 state 0x0 ref 1 00000100:00100000:3.0:1789762748.077157:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221518985580 time 221318985580 state 0x1 ref 1 00000100:00100000:0.0:1789762748.272078:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221518985580 time 221519145595 state 0x0 ref 1 00000100:00100000:0.0:1789762748.272118:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067289472:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762748.272172:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f17498f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.272209:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762748.272213:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762748.272220:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f17498f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.272248:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f17498f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.275537:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f17498f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.275549:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f17498f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.275557:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f17498f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.275563:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1bf2c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762748.275568:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f17498f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.275574:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f17498f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.275591:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f17498f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.275607:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8895c52/0xdf5a7ac59f17498f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762748.275661:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067289472:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3564us (198634us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762748.275668:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 149 00000100:00100000:0.0:1789762748.275674:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221518985580 time 221519145595 state 0x2 ref 0 00000100:00100000:0.0:1789762748.277677:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762748.277719:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067289600 00000100:00100000:3.0:1789762748.277729:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067289600 00000100:00100000:3.0:1789762748.277770:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221518985580 time 221519145595 state 0x0 ref 1 00000100:00100000:3.0:1789762748.277780:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221719145595 time 221519145595 state 0x1 ref 1 00010000:00010000:0.0:1789762748.475299:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174996 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.475310:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174996 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.475317:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174996 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.475322:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762748.475326:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174996 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.475333:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174996 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.475348:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174996 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762748.475360:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f174996 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762748.475418:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067289600:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3236us (197740us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762748.475425:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 150 00000100:00100000:0.0:1789762748.475432:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221719145595 time 221719230991 state 0x2 ref 0 00000100:00100000:0.0:1789762748.477411:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762748.672279:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221919230991 time 221919336613 state 0x0 ref 1 00000100:00100000:2.0:1789762748.672324:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067289728:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762748.672376:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f17499d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762748.672394:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762748.672399:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762748.672404:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f17499d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762748.672418:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f17499d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762748.675213:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f17499d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762748.675222:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f17499d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762748.675257:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f17499d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762748.675267:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36cb8e2c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762748.675273:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f17499d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762748.675285:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f17499d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762748.675303:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f17499d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762748.675309:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f17499d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762748.675366:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067289728:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3065us (197955us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762748.675373:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 151 00000100:00100000:2.0:1789762748.675378:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221919230991 time 221919336613 state 0x2 ref 0 00000100:00100000:2.0:1789762748.677358:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762748.677400:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067289856 00000100:00100000:0.0:1789762748.677411:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067289856 00000100:00100000:0.0:1789762748.677450:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 221919230991 time 221919336613 state 0x0 ref 1 00000100:00100000:0.0:1789762748.677461:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222119336613 time 221919336613 state 0x1 ref 1 00000100:00100000:2.0:1789762748.872345:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222119336613 time 222119412350 state 0x0 ref 1 00000100:00100000:2.0:1789762748.872382:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067289856:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762748.872451:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f1749a4 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762748.872470:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762748.872474:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762748.872480:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f1749a4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762748.872497:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074cacb9a/0xdf5a7ac59f1749a4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762748.877631:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067289984 00000100:00100000:0.0:1789762748.877642:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067289984 00000100:00100000:0.0:1789762748.877690:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222119336613 time 222119412350 state 0x0 ref 1 00000100:00100000:0.0:1789762748.877704:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222319412350 time 222119412350 state 0x1 ref 1 00000100:00100000:2.0:1789762748.928140:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff881c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074627456:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762748.928250:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762748.928303:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074627456 00000100:00100000:0.0:1789762748.928313:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074627456 00000100:00100000:0.0:1789762748.928364:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:0.0:1789762748.928374:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074627456:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762748.928418:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762748.928468:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074627456:12345-0@lo:41:osp-pre-1-0.0 Request processed in 97us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762748.928492:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00100000:2.0:1789762748.928529:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff881c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074627456:0@lo:41:osp-pre-1-0.0 00010000:00010000:2.0:1789762749.056105:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762749.072423:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222319412350 time 222319492463 state 0x0 ref 1 00000100:00100000:3.0:1789762749.072466:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067289984:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:2.0:1789762749.072487:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762749.072516:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749ab lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.072565:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762749.072571:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762749.072579:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749ab lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.072593:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749ab lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.075529:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749ab lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.075543:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749ab lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.075554:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749ab lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.075565:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762749.075571:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749ab lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.075581:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749ab lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.075601:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749ab lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.075619:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749ab lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762749.075677:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067289984:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3237us (198110us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762749.075685:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 153 00000100:00100000:3.0:1789762749.075690:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222319412350 time 222319492463 state 0x2 ref 0 00000100:00100000:0.0:1789762749.078271:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762749.078343:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067290112 00000100:00100000:2.0:1789762749.078354:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067290112 00000100:00100000:2.0:1789762749.078404:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222319412350 time 222319492463 state 0x0 ref 1 00000100:00100000:2.0:1789762749.078417:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222519492463 time 222319492463 state 0x1 ref 1 00000100:00100000:3.0:1789762749.272559:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222519492463 time 222519626237 state 0x0 ref 1 00000100:00100000:3.0:1789762749.272599:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee8c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067290112:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762749.272653:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749b2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.272670:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762749.272674:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762749.272680:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749b2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.272694:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749b2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.275644:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749b2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.275672:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749b2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.275679:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749b2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.275684:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762749.275688:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749b2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.275697:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749b2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.275713:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749b2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762749.275718:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749b2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762749.275773:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee8c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067290112:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3196us (197503us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762749.275780:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 154 00000100:00100000:3.0:1789762749.275786:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222519492463 time 222519626237 state 0x2 ref 0 00000100:00100000:2.0:1789762749.277879:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067290240 00000100:00100000:2.0:1789762749.277889:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067290240 00000100:00100000:2.0:1789762749.277952:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222519492463 time 222519626237 state 0x0 ref 1 00000100:00100000:2.0:1789762749.277963:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222719626237 time 222519626237 state 0x1 ref 1 00010000:00010000:0.0:1789762749.476007:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1749b9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.476021:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1749b9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.476033:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1749b9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.476042:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762749.476048:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1749b9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.476061:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1749b9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.476085:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1749b9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.476092:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1749b9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762749.476152:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067290240:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3462us (198328us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762749.476171:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 155 00000100:00100000:0.0:1789762749.476178:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222719626237 time 222719737892 state 0x2 ref 0 00000100:00100000:0.0:1789762749.478118:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762749.478158:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067290368 00000100:00100000:3.0:1789762749.478175:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067290368 00000100:00100000:3.0:1789762749.478215:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222719626237 time 222719737892 state 0x0 ref 1 00000100:00100000:3.0:1789762749.478240:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222919737892 time 222719737892 state 0x1 ref 1 00000100:00100000:0.0:1789762749.672770:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222919737892 time 222919836498 state 0x0 ref 1 00000100:00100000:0.0:1789762749.672815:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067290368:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762749.672867:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.672887:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762749.672891:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762749.672897:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.672911:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.676209:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.676246:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.676257:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.676280:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762749.676285:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.676293:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.676308:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.676314:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762749.676387:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067290368:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3589us (198267us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762749.676395:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 156 00000100:00100000:0.0:1789762749.676402:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222919737892 time 222919836498 state 0x2 ref 0 00000100:00100000:0.0:1789762749.678712:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762749.678756:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067290496 00000100:00100000:3.0:1789762749.678765:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067290496 00000100:00100000:3.0:1789762749.678802:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 222919737892 time 222919836498 state 0x0 ref 1 00000100:00100000:3.0:1789762749.678812:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223119836498 time 222919836498 state 0x1 ref 1 00000100:00100000:0.0:1789762749.872873:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223119836498 time 223119944263 state 0x0 ref 1 00000100:00100000:0.0:1789762749.872908:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067290496:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762749.872966:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.873003:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762749.873006:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762749.873011:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762749.873025:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1749c7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762749.878690:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067290624 00000100:00100000:0.0:1789762749.878698:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067290624 00000100:00100000:0.0:1789762749.878732:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223119836498 time 223119944263 state 0x0 ref 1 00000100:00100000:0.0:1789762749.878743:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223319944263 time 223119944263 state 0x1 ref 1 00000100:00100000:2.0:1789762749.952320:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074627584 00000100:00100000:2.0:1789762749.952344:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074627584 00000100:00100000:2.0:1789762749.952416:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:2.0:1789762749.952429:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074627584:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762749.952475:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762749.952562:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074627584:12345-0@lo:41:osp-pre-0-1.0 Request processed in 135us (349us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762749.952578:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:0.0:1789762750.072950:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223319944263 time 223320023292 state 0x0 ref 1 00000100:00100000:0.0:1789762750.072975:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3beea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067290624:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762750.073012:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f1749ce lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762750.073027:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762750.073031:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762750.073036:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f1749ce lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762750.073047:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f1749ce lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762750.076195:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f1749ce lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762750.076207:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f1749ce lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762750.076233:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f1749ce lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762750.076242:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762750.076247:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f1749ce lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762750.076257:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f1749ce lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762750.076276:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f1749ce lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762750.076285:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f1749ce lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762750.076336:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3beea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067290624:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3370us (197697us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762750.076344:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 158 00000100:00100000:0.0:1789762750.076355:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223319944263 time 223320023292 state 0x2 ref 0 00000100:00100000:2.0:1789762750.078620:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762750.078659:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067290752 00000100:00100000:0.0:1789762750.078667:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067290752 00000100:00100000:0.0:1789762750.078698:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223319944263 time 223320023292 state 0x0 ref 1 00000100:00100000:0.0:1789762750.078709:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223520023292 time 223320023292 state 0x1 ref 1 00000100:00100000:0.0:1789762750.278303:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762750.473146:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223720115925 time 223720211757 state 0x0 ref 1 00000100:00100000:3.0:1789762750.473184:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067290880:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762750.473259:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749dc lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.473282:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762750.473287:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762750.473296:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749dc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.473314:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749dc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.476801:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749dc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.476818:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749dc lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.476835:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749dc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.476841:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762750.476844:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749dc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.476850:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749dc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.476866:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749dc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.476872:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749dc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762750.476971:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067290880:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3800us (198666us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762750.476979:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 160 00000100:00100000:3.0:1789762750.476987:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223720115925 time 223720211757 state 0x2 ref 0 00000100:00100000:2.0:1789762750.478986:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762750.479045:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067291008 00000100:00100000:0.0:1789762750.479057:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067291008 00000100:00100000:0.0:1789762750.479109:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223720115925 time 223720211757 state 0x0 ref 1 00000100:00100000:0.0:1789762750.479123:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223920211757 time 223720211757 state 0x1 ref 1 00000100:00100000:3.0:1789762750.673209:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223920211757 time 223920281470 state 0x0 ref 1 00000100:00100000:3.0:1789762750.673247:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067291008:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762750.673284:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749e3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.673306:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762750.673309:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762750.673314:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749e3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.673325:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749e3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.677087:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749e3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.677119:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749e3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.677128:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749e3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.677136:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762750.677140:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749e3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.677149:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749e3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.677167:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749e3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762750.677190:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749e3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762750.677256:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067291008:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4019us (198271us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762750.677266:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 161 00000100:00100000:3.0:1789762750.677273:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 223920211757 time 223920281470 state 0x2 ref 0 00000100:00100000:2.0:1789762750.679440:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762750.873328:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 224120281470 time 224120397377 state 0x0 ref 1 00000100:00100000:0.0:1789762750.873362:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067291136:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762750.873409:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f1749ea lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762750.873427:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762750.873433:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762750.873439:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f1749ea lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762750.873457:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f1749ea lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762750.881056:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762750.881222:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067291264 00000100:00100000:3.0:1789762750.881232:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067291264 00000100:00100000:3.0:1789762750.881283:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 224120281470 time 224120397377 state 0x0 ref 1 00000100:00100000:3.0:1789762750.881297:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 224320397377 time 224120397377 state 0x1 ref 1 00000100:00100000:0.0:1789762750.919896:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762750.919962:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067291392 00000100:00100000:2.0:1789762750.919970:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067291392 00000100:00100000:0.0:1789762750.919987:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762750.920021:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 313 00000100:00100000:2.0:1789762750.920031:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb938450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067291392:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762750.920045:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762750.920076:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067291648 00000100:00100000:3.0:1789762750.920082:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067291648 00000100:00100000:2.0:1789762750.920095:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb938450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067291392:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 64us (206us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762750.920106:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 313 00000100:00100000:0.0:1789762750.920112:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762750.920129:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067291776 00000100:00100000:3.0:1789762750.920133:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067291776 00000100:00080000:2.0:1789762750.920144:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762750.920151:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762750.920174:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 259 00000100:00100000:3.0:1789762750.920185:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 260 00000100:00100000:2.0:1789762750.920191:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2731500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067291648:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762750.920200:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2731880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067291776:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762750.920246:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2731500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067291648:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 61us (204us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762750.920260:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762750.920266:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 259 00000100:00100000:2.0:1789762750.920286:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067291904 00000100:00100000:2.0:1789762750.920298:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067291904 00000100:00100000:3.0:1789762750.920304:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2731880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067291776:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 102us (192us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762750.920312:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 260 00000100:00100000:3.0:1789762750.920335:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 261 00000100:00100000:3.0:1789762750.920343:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2731c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067291904:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:2.0:1789762750.920349:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762750.920370:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2731c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067291904:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 28us (114us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762750.920384:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 261 00000100:00100000:3.0:1789762751.073380:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 224320397377 time 224320451715 state 0x0 ref 1 00000100:00100000:3.0:1789762751.073418:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067291264:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:2.0:1789762751.073441:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762751.073458:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749f1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.073473:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762751.073476:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762751.073481:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749f1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.073492:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749f1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.077006:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749f1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.077017:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749f1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.077027:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749f1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.077037:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762751.077042:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749f1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.077062:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749f1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.077083:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749f1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.077092:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f1749f1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762751.077153:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067291264:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3755us (196103us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762751.077162:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 163 00000100:00100000:3.0:1789762751.077168:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 224320397377 time 224320451715 state 0x2 ref 0 00000100:00100000:0.0:1789762751.079576:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762751.079639:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067292032 00000100:00100000:3.0:1789762751.079649:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067292032 00000100:00100000:3.0:1789762751.079692:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 224320397377 time 224320451715 state 0x0 ref 1 00000100:00100000:3.0:1789762751.079706:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 224520451715 time 224320451715 state 0x1 ref 1 00010000:00010000:0.0:1789762751.104116:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762751.273465:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 224520451715 time 224520536243 state 0x0 ref 1 00000100:00100000:0.0:1789762751.273494:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2732d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067292032:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762751.273531:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f1749f8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762751.273545:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762751.273548:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762751.273554:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f1749f8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762751.273565:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f1749f8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762751.278702:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762751.473610:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 224720536243 time 224720676122 state 0x0 ref 1 00000100:00100000:0.0:1789762751.473651:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067292160:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762751.473716:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1749ff lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762751.473746:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762751.473777:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762751.473788:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1749ff lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762751.473809:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1749ff lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762751.480174:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1749ff lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762751.480227:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1749ff lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762751.480263:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1749ff lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762751.480282:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762751.480291:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1749ff lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762751.480310:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1749ff lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762751.480352:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1749ff lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762751.480363:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000639239ff/0xdf5a7ac59f1749ff lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762751.480455:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067292160:12345-192.168.203.37@tcp:3:dd.500 Request processed in 6819us (201750us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762751.480486:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 165 00000100:00100000:2.0:1789762751.480497:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 224720536243 time 224720676122 state 0x2 ref 0 00000100:00100000:2.0:1789762751.482865:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762751.482962:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067292288 00000100:00100000:3.0:1789762751.482980:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067292288 00000100:00100000:3.0:1789762751.483037:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 224720536243 time 224720676122 state 0x0 ref 1 00000100:00100000:3.0:1789762751.483052:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 224920676122 time 224720676122 state 0x1 ref 1 00000100:00100000:0.0:1789762751.681821:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762751.873859:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225120821885 time 225120930068 state 0x0 ref 1 00000100:00100000:3.0:1789762751.873910:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2730700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067292416:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762751.873979:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a0d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.873998:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762751.874003:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762751.874011:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a0d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.874029:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a0d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.877036:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a0d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.877048:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a0d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.877057:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a0d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.877066:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762751.877072:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a0d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.877081:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a0d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.877106:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a0d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762751.877115:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a0d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762751.877171:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2730700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067292416:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3273us (195355us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762751.877181:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 167 00000100:00100000:3.0:1789762751.877188:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225120821885 time 225120930068 state 0x2 ref 0 00000100:00100000:2.0:1789762751.879241:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762751.879292:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067292544 00000100:00100000:0.0:1789762751.879300:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067292544 00000100:00100000:0.0:1789762751.879332:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225120821885 time 225120930068 state 0x0 ref 1 00000100:00100000:0.0:1789762751.879341:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225320930068 time 225120930068 state 0x1 ref 1 00000100:00100000:0.0:1789762752.000106:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff882d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074627968:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762752.000177:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff880700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074628224:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.000327:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff883800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074628352:0@lo:400:kworker.0 00000100:00100000:0.0:1789762752.000354:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762752.000373:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762752.000387:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762752.000425:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2733800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074627712:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762752.000429:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff883100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074628864:0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.000436:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff882a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074628736:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.000460:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762752.000478:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762752.000485:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762752.000486:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8ab100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074629376:0@lo:400:kworker.0 00000100:00100000:0.0:1789762752.000504:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074627840:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762752.000515:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff882680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074629248:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.000524:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762752.000549:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762752.000565:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762752.000574:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074627712 00000100:00100000:3.0:1789762752.000580:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074627712 00000100:00100000:0.0:1789762752.000590:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff881f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074628480:0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.000597:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee70ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074629760:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.000612:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 111 00000100:00100000:3.0:1789762752.000623:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2732300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074627712:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762752.000634:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:3.0:1789762752.000637:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762752.000647:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762752.000676:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074628224 00000100:00100000:0.0:1789762752.000682:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff883b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074628992:0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.000682:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074628224 00000100:00100000:3.0:1789762752.000683:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2732300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074627712:12345-0@lo:13:osp-pre-1-0.0 Request processed in 62us (205us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762752.000707:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 111 00000100:00100000:2.0:1789762752.000722:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 314 00000100:00100000:2.0:1789762752.000733:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb93bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074628224:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.000737:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074629376 00000100:00100000:0.0:1789762752.000742:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762752.000745:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074629376 00000100:00100000:0.0:1789762752.000769:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e4ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074629504:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.000785:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074629760 00000100:00100000:2.0:1789762752.000786:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb93bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074628224:12345-0@lo:400:kworker.0 Request processed in 54us (414us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762752.000790:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074629760 00000100:00100000:2.0:1789762752.000797:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 314 00000100:00100000:0.0:1789762752.000802:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762752.000825:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074629504 00000100:00100000:3.0:1789762752.000830:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074629504 00000100:00080000:2.0:1789762752.000837:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762752.000843:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762752.000856:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 105 00000100:00100000:2.0:1789762752.000861:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074628736 00000100:00100000:0.0:1789762752.000865:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff882d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074627968:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762752.000867:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee70d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074629376:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.000875:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074628736 00000100:00100000:0.0:1789762752.000888:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2733800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074627712:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762752.000908:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074627840:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762752.000911:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074628480 00000100:00100000:2.0:1789762752.000915:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074628480 00000100:00100000:3.0:1789762752.000946:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee70d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074629376:12345-0@lo:400:kworker.0 Request processed in 82us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762752.000954:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 105 00000100:00100000:0.0:1789762752.000963:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:0.0:1789762752.000973:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff881c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074628608:0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.000981:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:3.0:1789762752.000981:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 106 00000100:00100000:3.0:1789762752.000993:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074629760:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.001000:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074628736:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762752.001009:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762752.001030:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff880000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074629120:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.001046:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074629760:12345-0@lo:400:kworker.0 Request processed in 56us (412us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762752.001055:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 106 00000100:00100000:2.0:1789762752.001069:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074628736:12345-0@lo:400:kworker.0 Request processed in 71us (580us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762752.001073:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762752.001084:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 107 00000100:00100000:2.0:1789762752.001086:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00100000:0.0:1789762752.001092:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee70ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074629632:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.001100:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2733480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074629504:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.001107:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:2.0:1789762752.001118:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2730000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074628480:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762752.001138:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762752.001153:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2733480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074629504:12345-0@lo:400:kworker.0 Request processed in 57us (355us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762752.001159:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 107 00000100:00100000:2.0:1789762752.001169:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2730000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074628480:12345-0@lo:400:kworker.0 Request processed in 53us (522us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762752.001176:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074629632 00000100:00100000:2.0:1789762752.001177:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00100000:3.0:1789762752.001182:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074629632 00000100:00100000:0.0:1789762752.001183:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff881f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074628480:0@lo:400:kworker.0 00000100:00100000:0.0:1789762752.001217:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e4ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074629504:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.001238:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 108 00000100:00100000:3.0:1789762752.001248:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2730a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074629632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762752.001271:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 264 00000100:00080000:2.0:1789762752.001272:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762752.001282:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762752.001287:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074629248:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762752.001290:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762752.001315:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2730a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074629632:12345-0@lo:400:kworker.0 Request processed in 67us (180us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762752.001321:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff882a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074628736:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.001325:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 108 00000100:00100000:0.0:1789762752.001333:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074629248:12345-0@lo:400:kworker.0 Request processed in 50us (770us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762752.001341:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 264 00000100:00100000:2.0:1789762752.001353:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff882680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074629248:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.001363:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 266 00000100:00100000:0.0:1789762752.001370:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 267 00000100:00100000:3.0:1789762752.001375:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2733800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074628608:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.001378:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee70ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074629760:0@lo:400:kworker.0 00000100:00100000:0.0:1789762752.001383:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2731180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074629120:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.001401:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff880700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074628224:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.001430:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2733800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074628608:12345-0@lo:400:kworker.0 Request processed in 57us (421us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762752.001439:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2731180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074629120:12345-0@lo:400:kworker.0 Request processed in 59us (373us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762752.001444:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 266 00000100:00100000:0.0:1789762752.001451:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 267 00000100:00100000:3.0:1789762752.001481:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074628096 00000100:00100000:3.0:1789762752.001485:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074628096 00000100:00100000:2.0:1789762752.001492:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff883800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074628352:0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.001525:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff883100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074628864:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.001527:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 113 00000100:00100000:3.0:1789762752.001542:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074628096:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762752.001554:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762752.001562:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8ab100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074629376:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.001609:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074628096:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (180us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762752.001617:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 113 00000100:00100000:2.0:1789762752.001651:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074628096:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762752.073909:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225320930068 time 225320980700 state 0x0 ref 1 00000100:00100000:0.0:1789762752.073977:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067292544:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762752.074018:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004ba4b360/0xdf5a7ac59f174a14 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.074035:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762752.074039:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762752.074045:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ba4b360/0xdf5a7ac59f174a14 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.074058:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ba4b360/0xdf5a7ac59f174a14 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762752.077150:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004ba4b360/0xdf5a7ac59f174a14 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762752.077160:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ba4b360/0xdf5a7ac59f174a14 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762752.077167:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ba4b360/0xdf5a7ac59f174a14 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762752.077172:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c7278400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762752.077182:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ba4b360/0xdf5a7ac59f174a14 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762752.077189:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ba4b360/0xdf5a7ac59f174a14 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762752.077216:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ba4b360/0xdf5a7ac59f174a14 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762752.077222:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ba4b360/0xdf5a7ac59f174a14 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762752.077275:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067292544:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3308us (198033us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762752.077285:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 168 00000100:00100000:3.0:1789762752.077292:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225320930068 time 225320980700 state 0x2 ref 0 00000100:00100000:2.0:1789762752.079371:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:2.0:1789762752.128017:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762752.274052:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225520980700 time 225521123212 state 0x0 ref 1 00000100:00100000:0.0:1789762752.274081:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067292672:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762752.274120:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a1b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.274134:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762752.274138:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762752.274143:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a1b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.274154:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a1b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.277139:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a1b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.277147:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a1b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.277154:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a1b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.277158:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c7279000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762752.277162:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a1b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.277168:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a1b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.277182:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a1b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.277187:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a1b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762752.277246:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067292672:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3179us (197878us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762752.277254:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 169 00000100:00100000:0.0:1789762752.277264:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225520980700 time 225521123212 state 0x2 ref 0 00000100:00100000:2.0:1789762752.279454:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762752.279491:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067292800 00000100:00100000:3.0:1789762752.279501:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067292800 00000100:00100000:3.0:1789762752.279537:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225520980700 time 225521123212 state 0x0 ref 1 00000100:00100000:3.0:1789762752.279549:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225721123212 time 225521123212 state 0x1 ref 1 00000100:00100000:3.0:1789762752.474116:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225721123212 time 225721188008 state 0x0 ref 1 00000100:00100000:3.0:1789762752.474145:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067292800:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762752.474181:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a22 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762752.474223:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762752.474227:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762752.474232:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a22 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762752.474243:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a22 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.477081:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a22 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.477091:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a22 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.477098:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a22 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.477103:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762752.477106:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a22 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.477112:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a22 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.477125:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a22 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.477130:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174a22 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762752.477177:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067292800:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3045us (197723us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762752.477184:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 170 00000100:00100000:0.0:1789762752.477195:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225721123212 time 225721188008 state 0x2 ref 0 00000100:00100000:2.0:1789762752.479085:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762752.479122:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067292928 00000100:00100000:3.0:1789762752.479129:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067292928 00000100:00100000:3.0:1789762752.479156:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225721123212 time 225721188008 state 0x0 ref 1 00000100:00100000:3.0:1789762752.479165:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225921188008 time 225721188008 state 0x1 ref 1 00000100:00100000:0.0:1789762752.674244:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225921188008 time 225921316597 state 0x0 ref 1 00000100:00100000:0.0:1789762752.674274:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067292928:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762752.674311:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a29 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.674324:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762752.674328:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762752.674333:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a29 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.674344:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a29 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.677115:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a29 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.677126:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a29 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.677133:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a29 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.677138:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c7279000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762752.677141:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a29 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.677167:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a29 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.677181:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a29 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.677187:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a29 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762752.677246:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067292928:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2986us (198160us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762752.677253:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 171 00000100:00100000:0.0:1789762752.677258:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225921188008 time 225921316597 state 0x2 ref 0 00000100:00100000:2.0:1789762752.679180:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762752.679225:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067293056 00000100:00100000:0.0:1789762752.679233:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067293056 00000100:00100000:0.0:1789762752.679264:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 225921188008 time 225921316597 state 0x0 ref 1 00000100:00100000:0.0:1789762752.679273:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226121316597 time 225921316597 state 0x1 ref 1 00000100:00100000:0.0:1789762752.874322:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226121316597 time 226121393625 state 0x0 ref 1 00000100:00100000:0.0:1789762752.874349:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067293056:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762752.874386:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a30 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.874416:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762752.874420:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762752.874425:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a30 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.874438:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a30 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.878435:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a30 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.878465:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a30 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.878476:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a30 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.878486:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c7279000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762752.878491:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a30 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.878528:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a30 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.878546:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a30 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762752.878568:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6c3e5e7/0xdf5a7ac59f174a30 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762752.878654:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067293056:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4315us (199472us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762752.878665:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 172 00000100:00100000:0.0:1789762752.878672:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226121316597 time 226121393625 state 0x2 ref 0 00000100:00100000:3.0:1789762752.881032:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762752.881063:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067293184 00000100:00100000:0.0:1789762752.881071:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067293184 00000100:00100000:0.0:1789762752.881123:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226121316597 time 226121393625 state 0x0 ref 1 00000100:00100000:0.0:1789762752.881137:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226321393625 time 226121393625 state 0x1 ref 1 00000100:00080000:2.0:1789762753.074461:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789762753.078500:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e518dfe9/0xdf5a7ac59f174a37 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762753.078516:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e518dfe9/0xdf5a7ac59f174a37 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762753.078524:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e518dfe9/0xdf5a7ac59f174a37 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762753.078529:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1bf3400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762753.078533:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e518dfe9/0xdf5a7ac59f174a37 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762753.078539:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e518dfe9/0xdf5a7ac59f174a37 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762753.078555:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e518dfe9/0xdf5a7ac59f174a37 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762753.078560:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e518dfe9/0xdf5a7ac59f174a37 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762753.078619:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee70ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067293184:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4206us (197586us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762753.078637:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 173 00000100:00100000:2.0:1789762753.078644:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226321393625 time 226321467767 state 0x2 ref 0 00000100:00100000:3.0:1789762753.080794:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762753.080840:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067293312 00000100:00100000:0.0:1789762753.080849:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067293312 00000100:00100000:0.0:1789762753.080883:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226321393625 time 226321467767 state 0x0 ref 1 00000100:00100000:0.0:1789762753.080892:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226521467767 time 226321467767 state 0x1 ref 1 00010000:00010000:2.0:1789762753.152029:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762753.274469:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226521467767 time 226521541603 state 0x0 ref 1 00000100:00100000:3.0:1789762753.274497:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067293312:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762753.274534:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e4c428e9/0xdf5a7ac59f174a3e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.274547:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762753.274551:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762753.274555:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4c428e9/0xdf5a7ac59f174a3e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.274584:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4c428e9/0xdf5a7ac59f174a3e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762753.279686:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762753.279739:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067293440 00000100:00100000:2.0:1789762753.279748:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067293440 00000100:00100000:2.0:1789762753.279781:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226521467767 time 226521541603 state 0x0 ref 1 00000100:00100000:2.0:1789762753.279790:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226721541603 time 226521541603 state 0x1 ref 1 00000100:00100000:3.0:1789762753.474654:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226721541603 time 226721719405 state 0x0 ref 1 00000100:00100000:3.0:1789762753.474710:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2733100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067293440:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762753.474765:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009b36363b/0xdf5a7ac59f174a45 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.474796:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762753.474800:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762753.474806:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b36363b/0xdf5a7ac59f174a45 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.474821:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b36363b/0xdf5a7ac59f174a45 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.478474:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009b36363b/0xdf5a7ac59f174a45 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.478502:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b36363b/0xdf5a7ac59f174a45 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.478534:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b36363b/0xdf5a7ac59f174a45 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.478562:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37fddbce00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762753.478569:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b36363b/0xdf5a7ac59f174a45 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.478578:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b36363b/0xdf5a7ac59f174a45 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.478601:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b36363b/0xdf5a7ac59f174a45 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.478609:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b36363b/0xdf5a7ac59f174a45 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762753.478706:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2733100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067293440:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4022us (199012us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762753.478723:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 175 00000100:00100000:3.0:1789762753.478735:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226721541603 time 226721719405 state 0x2 ref 0 00000100:00100000:3.0:1789762753.481294:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762753.481358:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067293568 00000100:00100000:2.0:1789762753.481369:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067293568 00000100:00100000:2.0:1789762753.481415:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226721541603 time 226721719405 state 0x0 ref 1 00000100:00100000:2.0:1789762753.481427:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 226921719405 time 226721719405 state 0x1 ref 1 00000100:00100000:3.0:1789762753.680388:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762753.874883:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227121861308 time 227121954717 state 0x0 ref 1 00000100:00100000:0.0:1789762753.874912:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067293696:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762753.874974:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f174a53 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762753.875006:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762753.875009:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762753.875014:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f174a53 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762753.875027:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f174a53 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.878065:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f174a53 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.878082:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f174a53 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.878100:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f174a53 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.878108:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762753.878112:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f174a53 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.878120:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f174a53 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.878141:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f174a53 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762753.878147:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f174a53 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762753.878235:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067293696:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3335us (197846us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762753.878243:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 177 00000100:00100000:3.0:1789762753.878251:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227121861308 time 227121954717 state 0x2 ref 0 00000100:00100000:3.0:1789762753.880104:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762753.880151:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067293824 00000100:00100000:0.0:1789762753.880160:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067293824 00000100:00100000:0.0:1789762753.880207:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227121861308 time 227121954717 state 0x0 ref 1 00000100:00100000:0.0:1789762753.880217:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227321954717 time 227121954717 state 0x1 ref 1 00000100:00100000:3.0:1789762754.048158:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074629888:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762754.048286:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762754.048338:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074629888 00000100:00100000:2.0:1789762754.048348:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074629888 00000100:00100000:2.0:1789762754.048412:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:2.0:1789762754.048428:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074629888:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762754.048462:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762754.048532:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074629888:12345-0@lo:41:osp-pre-1-0.0 Request processed in 106us (246us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762754.048540:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:3.0:1789762754.048588:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074629888:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762754.074952:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227321954717 time 227322025587 state 0x0 ref 1 00000100:00100000:0.0:1789762754.074972:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067293824:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762754.075007:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f174a5a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762754.075022:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762754.075026:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762754.075030:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f174a5a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762754.075040:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f174a5a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762754.080223:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762754.275089:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227522025587 time 227522158416 state 0x0 ref 1 00000100:00100000:2.0:1789762754.275124:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067293952:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762754.275166:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000adbe9e65/0xdf5a7ac59f174a61 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762754.275236:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762754.275242:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762754.275251:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adbe9e65/0xdf5a7ac59f174a61 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762754.275270:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adbe9e65/0xdf5a7ac59f174a61 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762754.279491:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000adbe9e65/0xdf5a7ac59f174a61 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762754.279506:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adbe9e65/0xdf5a7ac59f174a61 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762754.279516:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adbe9e65/0xdf5a7ac59f174a61 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762754.279524:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762754.279529:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adbe9e65/0xdf5a7ac59f174a61 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762754.279537:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adbe9e65/0xdf5a7ac59f174a61 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762754.279556:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adbe9e65/0xdf5a7ac59f174a61 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762754.279563:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adbe9e65/0xdf5a7ac59f174a61 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762754.279628:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067293952:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4520us (199409us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762754.279638:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 179 00000100:00100000:0.0:1789762754.279653:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227522025587 time 227522158416 state 0x2 ref 0 00000100:00100000:3.0:1789762754.281888:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762754.281974:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067294080 00000100:00100000:0.0:1789762754.281985:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067294080 00000100:00100000:0.0:1789762754.282024:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227522025587 time 227522158416 state 0x0 ref 1 00000100:00100000:0.0:1789762754.282036:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227722158416 time 227522158416 state 0x1 ref 1 00010000:00010000:2.0:1789762754.479486:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f174a68 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762754.479500:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f174a68 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762754.479509:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f174a68 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762754.479515:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762754.479519:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f174a68 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762754.479527:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f174a68 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762754.479542:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f174a68 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762754.479564:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f174a68 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762754.479631:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067294080:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4273us (197740us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762754.479639:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 180 00000100:00100000:2.0:1789762754.479645:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227722158416 time 227722365179 state 0x2 ref 0 00000100:00100000:3.0:1789762754.481806:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762754.675482:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227922365179 time 227922550175 state 0x0 ref 1 00000100:00100000:3.0:1789762754.675541:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067294208:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762754.675596:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a6f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.675619:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762754.675625:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762754.675633:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a6f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.675652:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a6f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.680569:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a6f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.680588:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a6f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.680600:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a6f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.680610:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762754.680633:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a6f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.680646:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a6f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.680668:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a6f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.680678:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a6f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762754.680762:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067294208:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5255us (198961us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762754.680774:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 181 00000100:00100000:3.0:1789762754.680783:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227922365179 time 227922550175 state 0x2 ref 0 00000100:00100000:3.0:1789762754.684333:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762754.684431:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067294336 00000100:00100000:0.0:1789762754.684445:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067294336 00000100:00100000:0.0:1789762754.684496:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 227922365179 time 227922550175 state 0x0 ref 1 00000100:00100000:0.0:1789762754.684511:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228122550175 time 227922550175 state 0x1 ref 1 00000100:00100000:3.0:1789762754.875625:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228122550175 time 228122692216 state 0x0 ref 1 00000100:00100000:3.0:1789762754.875661:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067294336:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762754.875721:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a76 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.875749:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762754.875757:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762754.875769:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a76 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.875824:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a76 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.879855:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a76 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.879867:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a76 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.879875:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a76 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.879881:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762754.879886:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a76 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.879894:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a76 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.879910:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a76 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762754.879937:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a76 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762754.880015:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067294336:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4370us (195686us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762754.880026:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 182 00000100:00100000:3.0:1789762754.880033:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228122550175 time 228122692216 state 0x2 ref 0 00000100:00100000:3.0:1789762754.882120:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762754.882227:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067294464 00000100:00100000:2.0:1789762754.882239:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067294464 00000100:00100000:2.0:1789762754.882286:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228122550175 time 228122692216 state 0x0 ref 1 00000100:00100000:2.0:1789762754.882300:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228322692216 time 228122692216 state 0x1 ref 1 00000100:00100000:2.0:1789762755.072257:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789762755.072295:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074630016:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762755.072400:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762755.072471:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074630016 00000100:00100000:0.0:1789762755.072484:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074630016 00000100:00100000:0.0:1789762755.072561:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:0.0:1789762755.072579:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074630016:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762755.072627:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762755.072733:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074630016:12345-0@lo:41:osp-pre-0-1.0 Request processed in 157us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762755.072747:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:2.0:1789762755.072833:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074630016:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789762755.075761:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228322692216 time 228322831609 state 0x0 ref 1 00000100:00100000:0.0:1789762755.075825:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067294464:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:2.0:1789762755.075849:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762755.075893:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f174a7d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762755.075978:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762755.076005:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762755.076016:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f174a7d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762755.076039:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f174a7d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762755.080484:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f174a7d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762755.080503:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f174a7d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762755.080514:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f174a7d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762755.080527:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5871000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762755.080536:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f174a7d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762755.080570:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f174a7d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762755.080600:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f174a7d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762755.080615:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f174a7d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762755.080717:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067294464:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4931us (198597us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762755.080733:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 183 00000100:00100000:0.0:1789762755.080743:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228322692216 time 228322831609 state 0x2 ref 0 00000100:00100000:0.0:1789762755.083274:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762755.083379:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067294592 00000100:00100000:3.0:1789762755.083393:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067294592 00000100:00100000:3.0:1789762755.083449:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228322692216 time 228322831609 state 0x0 ref 1 00000100:00100000:3.0:1789762755.083469:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228522831609 time 228322831609 state 0x1 ref 1 00010000:00010000:0.0:1789762755.200046:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762755.275815:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228522831609 time 228522887635 state 0x0 ref 1 00000100:00100000:3.0:1789762755.275844:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2730380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067294592:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762755.275882:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a84 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.275897:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762755.275900:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762755.275905:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a84 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.275933:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a84 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.278866:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a84 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.278877:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a84 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.278884:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a84 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.278890:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762755.278893:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a84 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.278899:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a84 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.278912:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a84 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.278933:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a84 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762755.278983:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2730380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067294592:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3153us (195714us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762755.278991:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 184 00000100:00100000:3.0:1789762755.279002:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228522831609 time 228522887635 state 0x2 ref 0 00000100:00100000:3.0:1789762755.281108:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067294720 00000100:00100000:3.0:1789762755.281116:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067294720 00000100:00100000:3.0:1789762755.281146:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228522831609 time 228522887635 state 0x0 ref 1 00000100:00100000:3.0:1789762755.281154:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228722887635 time 228522887635 state 0x1 ref 1 00000100:00100000:0.0:1789762755.481344:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067294848 00000100:00100000:0.0:1789762755.481352:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067294848 00000100:00100000:0.0:1789762755.481382:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228722887635 time 228723033983 state 0x0 ref 1 00000100:00100000:0.0:1789762755.481391:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228923033983 time 228723033983 state 0x1 ref 1 00000100:00100000:3.0:1789762755.676049:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228923033983 time 228923121730 state 0x0 ref 1 00000100:00100000:3.0:1789762755.676078:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067294848:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762755.676116:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a92 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.676130:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762755.676133:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762755.676138:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a92 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.676148:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a92 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.679198:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a92 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.679209:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a92 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.679215:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a92 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.679228:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762755.679231:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a92 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.679238:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a92 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.679251:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a92 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762755.679256:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174a92 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762755.679308:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067294848:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3243us (198014us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762755.679316:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 186 00000100:00100000:3.0:1789762755.679321:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228923033983 time 228923121730 state 0x2 ref 0 00000100:00100000:0.0:1789762755.681251:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762755.681289:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067294976 00000100:00100000:3.0:1789762755.681297:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067294976 00000100:00100000:3.0:1789762755.681330:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 228923033983 time 228923121730 state 0x0 ref 1 00000100:00100000:3.0:1789762755.681339:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229123121730 time 228923121730 state 0x1 ref 1 00000100:00100000:2.0:1789762755.876160:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229123121730 time 229123233632 state 0x0 ref 1 00000100:00100000:2.0:1789762755.876205:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2732a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067294976:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762755.876243:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174a99 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762755.876257:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762755.876274:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762755.876282:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174a99 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762755.876300:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174a99 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762755.881811:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762756.040052:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762756.040089:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762756.040114:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762756.040137:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762756.040138:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067295232 00000100:00100000:3.0:1789762756.040145:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067295232 00000100:00100000:0.0:1789762756.040170:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067295616 00000100:00100000:2.0:1789762756.040185:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762756.040191:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067295616 00000100:00100000:3.0:1789762756.040211:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 315 00000100:00100000:2.0:1789762756.040223:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067295744 00000100:00100000:2.0:1789762756.040230:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067295744 00000100:00100000:0.0:1789762756.040236:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 268 00000100:00100000:3.0:1789762756.040242:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb93b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067295232:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762756.040250:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2396:x1876702067295360:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762756.040281:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 270 00000100:00100000:2.0:1789762756.040292:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067295488:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762756.040318:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2396:x1876702067295360:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 65us (267us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762756.040321:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb93b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067295232:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 78us (221us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762756.040335:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 268 00000100:00100000:2.0:1789762756.040338:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067295488:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 42us (194us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762756.040346:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 315 00000100:00100000:2.0:1789762756.040356:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 270 00000100:00100000:0.0:1789762756.040363:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 271 00000100:00100000:0.0:1789762756.040373:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067295744:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:2.0:1789762756.040383:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762756.040388:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762756.040403:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067295744:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 32us (229us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762756.040416:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 271 00000100:00100000:2.0:1789762756.076255:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229323233632 time 229323328185 state 0x0 ref 1 00000100:00100000:2.0:1789762756.076278:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2730e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067295104:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762756.076315:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174aa0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762756.076343:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762756.076347:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762756.076352:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174aa0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762756.076362:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174aa0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762756.079573:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174aa0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762756.079585:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174aa0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762756.079592:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174aa0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762756.079597:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5861a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762756.079600:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174aa0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762756.079606:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174aa0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762756.079619:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174aa0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762756.079637:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f174aa0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762756.079694:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2730e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067295104:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3422us (197881us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762756.079704:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 188 00000100:00100000:2.0:1789762756.079710:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229323233632 time 229323328185 state 0x2 ref 0 00000100:00100000:2.0:1789762756.082166:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762756.082222:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067295872 00000100:00100000:3.0:1789762756.082232:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067295872 00000100:00100000:3.0:1789762756.082271:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229323233632 time 229323328185 state 0x0 ref 1 00000100:00100000:3.0:1789762756.082283:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229523328185 time 229323328185 state 0x1 ref 1 00010000:00010000:2.0:1789762756.224041:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762756.276412:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229523328185 time 229523480509 state 0x0 ref 1 00000100:00100000:0.0:1789762756.276447:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067295872:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762756.276513:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aa7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.276535:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762756.276541:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762756.276550:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aa7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.276569:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aa7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.280386:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aa7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.280402:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aa7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.280418:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aa7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.280427:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca2e2600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762756.280448:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aa7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.280476:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aa7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.280499:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aa7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.280511:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aa7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762756.280580:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067295872:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4148us (198411us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762756.280589:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 189 00000100:00100000:0.0:1789762756.280596:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229523328185 time 229523480509 state 0x2 ref 0 00000100:00100000:0.0:1789762756.283111:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762756.476590:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229723480509 time 229723658465 state 0x0 ref 1 00000100:00100000:0.0:1789762756.476628:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067296000:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762756.476696:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aae lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.476712:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762756.476716:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762756.476722:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aae lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.476755:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ffdba620/0xdf5a7ac59f174aae lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762756.484047:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762756.484113:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067296128 00000100:00100000:2.0:1789762756.484123:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067296128 00000100:00100000:2.0:1789762756.484161:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229723480509 time 229723658465 state 0x0 ref 1 00000100:00100000:2.0:1789762756.484189:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229923658465 time 229723658465 state 0x1 ref 1 00000100:00100000:0.0:1789762756.676723:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229923658465 time 229923794855 state 0x0 ref 1 00000100:00100000:0.0:1789762756.676751:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067296128:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762756.676788:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174ab5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.676802:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762756.676806:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762756.676811:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174ab5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.676822:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174ab5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.679782:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174ab5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.679797:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174ab5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.679807:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174ab5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.679829:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca2e3200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762756.679834:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174ab5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.679843:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174ab5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.679862:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174ab5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.679871:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174ab5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762756.679955:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067296128:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3213us (195906us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762756.679965:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 191 00000100:00100000:0.0:1789762756.679972:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229923658465 time 229923794855 state 0x2 ref 0 00000100:00100000:3.0:1789762756.682083:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762756.682115:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067296256 00000100:00100000:0.0:1789762756.682124:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067296256 00000100:00100000:0.0:1789762756.682157:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 229923658465 time 229923794855 state 0x0 ref 1 00000100:00100000:0.0:1789762756.682166:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 230123794855 time 229923794855 state 0x1 ref 1 00000100:00100000:0.0:1789762756.876791:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 230123794855 time 230123862667 state 0x0 ref 1 00000100:00100000:0.0:1789762756.876817:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067296256:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762756.876863:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174abc lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.876877:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762756.876881:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762756.876885:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174abc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762756.876900:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029129bec/0xdf5a7ac59f174abc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762756.881785:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762757.076864:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 230323862667 time 230323935177 state 0x0 ref 1 00000100:00100000:3.0:1789762757.076906:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067296384:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:2.0:1789762757.076945:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762757.076969:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f174ac3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762757.076986:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762757.076991:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762757.076997:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f174ac3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762757.077012:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f174ac3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762757.082233:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762757.082281:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067296512 00000100:00100000:0.0:1789762757.082289:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067296512 00000100:00100000:0.0:1789762757.082322:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 230323862667 time 230323935177 state 0x0 ref 1 00000100:00100000:0.0:1789762757.082331:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 230523935177 time 230323935177 state 0x1 ref 1 00000100:00100000:2.0:1789762757.120048:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074630144:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762757.120111:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762757.120198:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff881c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074630656:0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.120233:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762757.120255:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff882680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074631168:0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.120281:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1789762757.120289:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762757.120306:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074631680:0@lo:400:kworker.0 00000100:00100000:3.0:1789762757.120308:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074630656 00000100:00100000:3.0:1789762757.120315:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074630656 00000100:00100000:3.0:1789762757.120351:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074631168 00000100:00100000:3.0:1789762757.120354:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074631168 00000100:00100000:0.0:1789762757.120365:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff883b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074630400:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762757.120378:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762757.120384:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 272 00000100:00100000:3.0:1789762757.120394:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074630656:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762757.120422:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762757.120443:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074630144:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762757.120457:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff883100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074630784:0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.120464:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:2.0:1789762757.120473:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff880000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074630528:0@lo:400:kworker.0 00000100:00100000:0.0:1789762757.120503:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762757.120509:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074630656:12345-0@lo:400:kworker.0 Request processed in 117us (275us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762757.120518:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762757.120519:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 272 00000100:00100000:0.0:1789762757.120527:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff882a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074631296:0@lo:400:kworker.0 00000100:00080000:3.0:1789762757.120545:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762757.120558:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff880700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074631040:0@lo:400:kworker.0 00000100:00100000:3.0:1789762757.120563:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074630400 00000100:00100000:3.0:1789762757.120568:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074630400 00000100:00100000:0.0:1789762757.120569:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762757.120590:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762757.120607:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074631808:0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.120616:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff882d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074631552:0@lo:400:kworker.0 00000100:00100000:0.0:1789762757.120652:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762757.120665:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762757.120702:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074632064:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762757.120711:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074630272 00000100:00100000:3.0:1789762757.120715:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074630272 00000100:00100000:2.0:1789762757.120736:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762757.120776:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074632064 00000100:00100000:2.0:1789762757.120777:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff881c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074630656:0@lo:400:kworker.0 00000100:00100000:0.0:1789762757.120789:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074632064 00000100:00100000:2.0:1789762757.120799:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff882680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074631168:0@lo:400:kworker.0 00000100:00100000:3.0:1789762757.120802:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 115 00000100:00100000:3.0:1789762757.120817:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074630400:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762757.120826:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 116 00002000:00000024:3.0:1789762757.120838:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762757.120841:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074630272:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789762757.120860:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762757.120861:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074630912 00000100:00100000:2.0:1789762757.120867:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074630912 00000100:00100000:3.0:1789762757.120882:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074630400:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (465us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762757.120893:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 115 00000100:00100000:2.0:1789762757.120899:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074631424 00000100:00100000:2.0:1789762757.120904:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074631424 00000100:00100000:0.0:1789762757.120946:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074630272:12345-0@lo:13:osp-pre-0-1.0 Request processed in 83us (387us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762757.120948:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 117 00000100:00100000:2.0:1789762757.120957:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 274 00000100:00100000:0.0:1789762757.120962:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 116 00000100:00100000:3.0:1789762757.120963:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074632064:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762757.120974:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074631296:12345-0@lo:400:kworker.0 00002000:00000024:3.0:1789762757.120977:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762757.120990:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074630784 00000100:00100000:0.0:1789762757.120996:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074630784 00000100:00100000:3.0:1789762757.121023:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074632064:12345-0@lo:13:osp-pre-1-1.0 Request processed in 65us (287us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762757.121029:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074631040 00000100:00100000:0.0:1789762757.121033:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074631040 00000100:00100000:2.0:1789762757.121039:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074631296:12345-0@lo:400:kworker.0 Request processed in 69us (471us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762757.121045:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 117 00000100:00100000:2.0:1789762757.121052:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 274 00000100:00100000:0.0:1789762757.121058:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:0.0:1789762757.121073:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074630784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762757.121075:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074631680 00000100:00100000:2.0:1789762757.121080:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:3.0:1789762757.121082:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074631680 00000100:00100000:2.0:1789762757.121092:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074631424:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762757.121115:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074631808 00000100:00100000:0.0:1789762757.121120:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074630784:12345-0@lo:400:kworker.0 Request processed in 51us (617us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762757.121121:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074631808 00000100:00100000:0.0:1789762757.121131:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00100000:0.0:1789762757.121153:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:3.0:1789762757.121154:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074631936 00000100:00100000:3.0:1789762757.121159:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074631936 00000100:00100000:2.0:1789762757.121160:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074631424:12345-0@lo:400:kworker.0 Request processed in 69us (449us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762757.121181:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074631040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.121187:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00000100:00100000:3.0:1789762757.121198:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 109 00000100:00100000:3.0:1789762757.121213:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074631680:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.121219:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 110 00000100:00100000:0.0:1789762757.121225:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074631040:12345-0@lo:400:kworker.0 Request processed in 63us (637us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762757.121231:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074631808:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762757.121234:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:3.0:1789762757.121260:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074631680:12345-0@lo:400:kworker.0 Request processed in 52us (889us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762757.121266:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 109 00000100:00100000:2.0:1789762757.121280:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074631808:12345-0@lo:400:kworker.0 Request processed in 53us (628us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762757.121286:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 111 00000100:00100000:2.0:1789762757.121293:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 110 00000100:00100000:3.0:1789762757.121297:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074631936:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.121338:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074630528 00000100:00100000:2.0:1789762757.121346:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074630528 00000100:00100000:3.0:1789762757.121348:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074631936:12345-0@lo:400:kworker.0 Request processed in 53us (567us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762757.121357:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 111 00000100:00100000:3.0:1789762757.121400:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:3.0:1789762757.121412:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb93a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074630528:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762757.121439:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762757.121445:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762757.121450:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762757.121451:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb93a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074630528:12345-0@lo:400:kworker.0 Request processed in 42us (935us total) trans 0 rc 0/0 00000100:00080000:2.0:1789762757.121456:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762757.121461:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00100000:2.0:1789762757.121510:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff880000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074630528:0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.121534:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff880700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074631040:0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.121561:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff882d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074631552:0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.121585:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074632064:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762757.121602:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074631680:0@lo:400:kworker.0 00010000:00010000:2.0:1789762757.248033:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762757.282280:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762757.477069:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 230724035277 time 230724141305 state 0x0 ref 1 00000100:00100000:2.0:1789762757.477098:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067296640:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762757.477153:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000987d79e7/0xdf5a7ac59f174ad1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762757.477179:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762757.477183:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762757.477187:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000987d79e7/0xdf5a7ac59f174ad1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762757.477200:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000987d79e7/0xdf5a7ac59f174ad1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.479816:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000987d79e7/0xdf5a7ac59f174ad1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.479832:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000987d79e7/0xdf5a7ac59f174ad1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.479858:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000987d79e7/0xdf5a7ac59f174ad1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.479867:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1aff200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762757.479872:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000987d79e7/0xdf5a7ac59f174ad1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.479879:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000987d79e7/0xdf5a7ac59f174ad1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.479892:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000987d79e7/0xdf5a7ac59f174ad1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.479897:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000987d79e7/0xdf5a7ac59f174ad1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762757.479984:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067296640:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2900us (197705us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762757.479991:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 195 00000100:00100000:0.0:1789762757.479997:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 230724035277 time 230724141305 state 0x2 ref 0 00000100:00100000:2.0:1789762757.481901:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762757.677201:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 230924141305 time 230924273042 state 0x0 ref 1 00000100:00100000:3.0:1789762757.677228:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067296768:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762757.677264:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000beaf5dea/0xdf5a7ac59f174ad8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762757.677289:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762757.677293:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762757.677298:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000beaf5dea/0xdf5a7ac59f174ad8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762757.677310:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000beaf5dea/0xdf5a7ac59f174ad8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762757.680031:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000beaf5dea/0xdf5a7ac59f174ad8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762757.680039:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000beaf5dea/0xdf5a7ac59f174ad8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762757.680099:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000beaf5dea/0xdf5a7ac59f174ad8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762757.680109:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca2e2200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762757.680114:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000beaf5dea/0xdf5a7ac59f174ad8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762757.680124:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000beaf5dea/0xdf5a7ac59f174ad8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762757.680142:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000beaf5dea/0xdf5a7ac59f174ad8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762757.680182:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000beaf5dea/0xdf5a7ac59f174ad8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762757.680237:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067296768:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3020us (198335us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762757.680248:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 196 00000100:00100000:3.0:1789762757.680255:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 230924141305 time 230924273042 state 0x2 ref 0 00000100:00100000:3.0:1789762757.682229:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762757.877307:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231124273042 time 231124378279 state 0x0 ref 1 00000100:00100000:0.0:1789762757.877340:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067296896:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762757.877383:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174adf lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.877414:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762757.877419:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762757.877426:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174adf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.877444:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174adf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.880560:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174adf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.880574:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174adf lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.880586:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174adf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.880594:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762757.880600:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174adf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.880611:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174adf lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.880630:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174adf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762757.880653:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174adf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762757.880710:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067296896:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3384us (198481us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762757.880721:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 197 00000100:00100000:0.0:1789762757.880728:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231124273042 time 231124378279 state 0x2 ref 0 00000100:00100000:0.0:1789762757.882708:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762757.882757:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067297024 00000100:00100000:2.0:1789762757.882765:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067297024 00000100:00100000:2.0:1789762757.882799:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231124273042 time 231124378279 state 0x0 ref 1 00000100:00100000:2.0:1789762757.882811:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231324378279 time 231124378279 state 0x1 ref 1 00000100:00100000:0.0:1789762758.077392:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231324378279 time 231324463047 state 0x0 ref 1 00000100:00100000:0.0:1789762758.077417:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067297024:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762758.077457:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174ae6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762758.077475:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762758.077480:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762758.077487:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174ae6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762758.077502:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174ae6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762758.080288:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174ae6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762758.080299:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174ae6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762758.080305:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174ae6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762758.080310:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762758.080314:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174ae6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762758.080320:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174ae6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762758.080333:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174ae6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762758.080348:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba57d324/0xdf5a7ac59f174ae6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762758.080399:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067297024:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2992us (197691us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762758.080405:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 198 00000100:00100000:3.0:1789762758.080411:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231324378279 time 231324463047 state 0x2 ref 0 00000100:00100000:2.0:1789762758.082606:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762758.082640:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067297152 00000100:00100000:3.0:1789762758.082649:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067297152 00000100:00100000:3.0:1789762758.082681:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231324378279 time 231324463047 state 0x0 ref 1 00000100:00100000:3.0:1789762758.082693:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231524463047 time 231324463047 state 0x1 ref 1 00000100:00100000:0.0:1789762758.277649:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231524463047 time 231524722414 state 0x0 ref 1 00000100:00100000:0.0:1789762758.277680:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067297152:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762758.277743:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f174aed lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762758.277764:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762758.277770:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762758.277778:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f174aed lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762758.277795:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f174aed lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762758.283994:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762758.284050:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067297280 00000100:00100000:0.0:1789762758.284059:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067297280 00000100:00100000:0.0:1789762758.284097:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231524463047 time 231524722414 state 0x0 ref 1 00000100:00100000:0.0:1789762758.284107:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231724722414 time 231524722414 state 0x1 ref 1 00000100:00100000:3.0:1789762758.483198:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762758.483250:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067297408 00000100:00100000:0.0:1789762758.483259:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067297408 00000100:00100000:0.0:1789762758.483293:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231724722414 time 231724830145 state 0x0 ref 1 00000100:00100000:0.0:1789762758.483302:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 231924830145 time 231724830145 state 0x1 ref 1 00000100:00100000:0.0:1789762758.683010:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762758.877960:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232124938159 time 232125032749 state 0x0 ref 1 00000100:00100000:2.0:1789762758.877990:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067297536:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762758.878027:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f174b02 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762758.878042:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762758.878045:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762758.878062:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f174b02 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762758.878076:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f174b02 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762758.880820:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f174b02 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762758.880830:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f174b02 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762758.880837:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f174b02 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762758.880842:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762758.880845:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f174b02 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762758.880851:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f174b02 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762758.880864:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f174b02 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762758.880869:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f174b02 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762758.880950:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067297536:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2974us (197940us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762758.880958:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 202 00000100:00100000:2.0:1789762758.880963:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232124938159 time 232125032749 state 0x2 ref 0 00000100:00100000:2.0:1789762758.882746:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762759.078051:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232325032749 time 232325122434 state 0x0 ref 1 00000100:00100000:3.0:1789762759.078092:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067297664:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762759.078131:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000586b4ffb/0xdf5a7ac59f174b09 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789762759.078145:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762759.078148:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762759.078168:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762759.078175:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000586b4ffb/0xdf5a7ac59f174b09 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762759.078190:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000586b4ffb/0xdf5a7ac59f174b09 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.080872:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000586b4ffb/0xdf5a7ac59f174b09 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.080883:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000586b4ffb/0xdf5a7ac59f174b09 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.080901:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000586b4ffb/0xdf5a7ac59f174b09 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.080907:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c727fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762759.080911:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000586b4ffb/0xdf5a7ac59f174b09 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.080936:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000586b4ffb/0xdf5a7ac59f174b09 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.080951:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000586b4ffb/0xdf5a7ac59f174b09 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.080957:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000586b4ffb/0xdf5a7ac59f174b09 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762759.081006:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067297664:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2938us (198261us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762759.081014:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 203 00000100:00100000:0.0:1789762759.081019:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232325032749 time 232325122434 state 0x2 ref 0 00000100:00100000:2.0:1789762759.082748:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762759.168069:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d274c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074632192:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762759.168126:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762759.168177:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074632192 00000100:00100000:0.0:1789762759.168185:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074632192 00000100:00100000:0.0:1789762759.168234:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1789762759.168243:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074632192:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762759.168292:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762759.168339:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074632192:12345-0@lo:41:osp-pre-1-0.0 Request processed in 97us (212us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762759.168346:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:2.0:1789762759.168373:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d274c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074632192:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762759.278171:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232525122434 time 232525242142 state 0x0 ref 1 00000100:00100000:0.0:1789762759.278207:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067297792:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762759.278246:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b10 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.278265:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762759.278269:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762759.278273:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b10 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.278284:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b10 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.280951:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b10 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.280962:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b10 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.280969:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b10 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.280992:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f1200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762759.280996:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b10 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.281003:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b10 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.281015:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b10 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.281021:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b10 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762759.281068:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067297792:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2883us (198321us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762759.281075:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 204 00000100:00100000:0.0:1789762759.281080:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232525122434 time 232525242142 state 0x2 ref 0 00000100:00100000:3.0:1789762759.282903:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762759.282950:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067297920 00000100:00100000:2.0:1789762759.282966:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067297920 00000100:00100000:2.0:1789762759.282998:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232525122434 time 232525242142 state 0x0 ref 1 00000100:00100000:2.0:1789762759.283007:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232725242142 time 232525242142 state 0x1 ref 1 00010000:00010000:0.0:1789762759.296049:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762759.478241:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232725242142 time 232725313184 state 0x0 ref 1 00000100:00100000:0.0:1789762759.478272:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067297920:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762759.478317:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b17 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.478356:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762759.478361:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762759.478366:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b17 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.478378:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b17 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762759.481117:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b17 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762759.481129:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b17 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762759.481136:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b17 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762759.481143:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f1200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762759.481163:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b17 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762759.481172:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b17 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762759.481187:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b17 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762759.481196:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004500c45b/0xdf5a7ac59f174b17 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762759.481255:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067297920:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2997us (198353us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762759.481274:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 205 00000100:00100000:3.0:1789762759.481280:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232725242142 time 232725313184 state 0x2 ref 0 00000100:00100000:3.0:1789762759.483113:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762759.483159:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067298048 00000100:00100000:0.0:1789762759.483167:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067298048 00000100:00100000:0.0:1789762759.483198:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232725242142 time 232725313184 state 0x0 ref 1 00000100:00100000:0.0:1789762759.483207:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232925313184 time 232725313184 state 0x1 ref 1 00000100:00100000:0.0:1789762759.678313:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232925313184 time 232925385000 state 0x0 ref 1 00000100:00100000:0.0:1789762759.678339:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067298048:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762759.678376:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b1e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.678390:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762759.678394:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762759.678398:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b1e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.678409:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b1e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.681387:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b1e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.681398:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b1e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.681404:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b1e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.681409:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f0a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762759.681412:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b1e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.681418:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b1e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.681431:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b1e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762759.681435:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b1e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762759.681487:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067298048:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3159us (198373us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762759.681496:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 206 00000100:00100000:0.0:1789762759.681510:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232925313184 time 232925385000 state 0x2 ref 0 00000100:00100000:2.0:1789762759.683710:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762759.683752:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067298176 00000100:00100000:0.0:1789762759.683759:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067298176 00000100:00100000:0.0:1789762759.683788:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 232925313184 time 232925385000 state 0x0 ref 1 00000100:00100000:0.0:1789762759.683797:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 233125385000 time 232925385000 state 0x1 ref 1 00000100:00100000:2.0:1789762759.883912:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762759.883965:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067298304 00000100:00100000:3.0:1789762759.883978:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067298304 00000100:00100000:3.0:1789762759.884030:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 233125385000 time 233125467777 state 0x0 ref 1 00000100:00100000:3.0:1789762759.884042:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 233325467777 time 233125467777 state 0x1 ref 1 00000100:00100000:2.0:1789762760.084017:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762760.084069:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067298432 00000100:00100000:0.0:1789762760.084079:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067298432 00000100:00100000:0.0:1789762760.084127:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 233325467777 time 233325559610 state 0x0 ref 1 00000100:00100000:0.0:1789762760.084137:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 233525559610 time 233325559610 state 0x1 ref 1 00000100:00100000:3.0:1789762760.192088:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789762760.192109:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff70d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074632320:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762760.192193:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762760.192536:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff70d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074632320:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762760.278577:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 233525559610 time 233525648959 state 0x0 ref 1 00000100:00100000:2.0:1789762760.278608:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067298432:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762760.278651:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f174b33 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762760.278690:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762760.278693:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762760.278698:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f174b33 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762760.278709:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f174b33 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.281234:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f174b33 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.281245:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f174b33 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.281253:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f174b33 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.281259:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762760.281263:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f174b33 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.281269:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f174b33 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.281294:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f174b33 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.281300:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f174b33 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762760.281347:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067298432:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2755us (197337us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762760.281355:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 209 00000100:00100000:0.0:1789762760.281360:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 233525559610 time 233525648959 state 0x2 ref 0 00000100:00100000:2.0:1789762760.283434:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:2.0:1789762760.320019:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762760.478719:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 233725648959 time 233725790629 state 0x0 ref 1 00000100:00100000:0.0:1789762760.478747:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067298560:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762760.478784:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b3a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.478799:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762760.478802:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762760.478807:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b3a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.478818:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b3a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.481795:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b3a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.481805:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b3a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.481812:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b3a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.481817:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f0a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762760.481820:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b3a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.481826:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b3a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.481840:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b3a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762760.481845:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f174b3a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762760.481905:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067298560:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3171us (198471us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762760.481912:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 210 00000100:00100000:0.0:1789762760.481956:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 233725648959 time 233725790629 state 0x2 ref 0 00000100:00100000:3.0:1789762760.483988:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:1.0:1789762760.679006:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b41 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762760.679043:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b41 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762760.682066:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b41 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762760.682078:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b41 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762760.682086:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b41 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762760.682094:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762760.682100:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b41 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762760.682110:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b41 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762760.682135:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b41 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762760.682158:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b41 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762760.682240:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067298688:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3373us (198252us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762760.682248:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 211 00000100:00100000:1.0:1789762760.682254:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 233925790629 time 233925916880 state 0x2 ref 0 00000100:00100000:3.0:1789762760.684369:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762760.684418:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067298816 00000100:00100000:0.0:1789762760.684428:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067298816 00000100:00100000:0.0:1789762760.684471:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 233925790629 time 233925916880 state 0x0 ref 1 00000100:00100000:0.0:1789762760.684482:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234125916880 time 233925916880 state 0x1 ref 1 00000100:00100000:3.0:1789762760.878962:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234125916880 time 234126033889 state 0x0 ref 1 00000100:00100000:3.0:1789762760.878991:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067298816:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762760.879029:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f174b48 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762760.879043:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762760.879046:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762760.879052:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f174b48 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762760.879089:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f174b48 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762760.881947:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f174b48 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762760.881957:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f174b48 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762760.881963:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f174b48 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762760.881968:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762760.881971:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f174b48 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762760.881978:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f174b48 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762760.881990:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f174b48 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762760.881995:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f174b48 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762760.882051:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067298816:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3073us (197688us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762760.882058:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 212 00000100:00100000:3.0:1789762760.882063:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234125916880 time 234126033889 state 0x2 ref 0 00000100:00100000:3.0:1789762760.883914:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762760.883976:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067298944 00000100:00100000:0.0:1789762760.883983:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067298944 00000100:00100000:0.0:1789762760.884014:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234125916880 time 234126033889 state 0x0 ref 1 00000100:00100000:0.0:1789762760.884023:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234326033889 time 234126033889 state 0x1 ref 1 00000100:00100000:0.0:1789762761.079036:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234326033889 time 234326107950 state 0x0 ref 1 00000100:00100000:0.0:1789762761.079072:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067298944:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:2.0:1789762761.079095:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762761.079111:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ac8330c8/0xdf5a7ac59f174b4f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.079124:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762761.079128:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762761.079147:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ac8330c8/0xdf5a7ac59f174b4f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.079159:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ac8330c8/0xdf5a7ac59f174b4f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.082012:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ac8330c8/0xdf5a7ac59f174b4f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.082045:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ac8330c8/0xdf5a7ac59f174b4f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.082055:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ac8330c8/0xdf5a7ac59f174b4f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.082063:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d999c000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762761.082068:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ac8330c8/0xdf5a7ac59f174b4f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.082078:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ac8330c8/0xdf5a7ac59f174b4f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.082095:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ac8330c8/0xdf5a7ac59f174b4f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.082102:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ac8330c8/0xdf5a7ac59f174b4f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762761.082178:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067298944:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3126us (198263us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762761.082188:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 213 00000100:00100000:0.0:1789762761.082195:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234326033889 time 234326107950 state 0x2 ref 0 00000100:00100000:0.0:1789762761.084179:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762761.084238:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067299072 00000100:00100000:1.0:1789762761.084252:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067299072 00000100:00100000:1.0:1789762761.084302:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234326033889 time 234326107950 state 0x0 ref 1 00000100:00100000:1.0:1789762761.084313:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234526107950 time 234326107950 state 0x1 ref 1 00000100:00100000:3.0:1789762761.160084:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762761.160118:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762761.160128:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067299200 00000100:00100000:2.0:1789762761.160149:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067299200 00000100:00100000:0.0:1789762761.160173:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067299328 00000100:00100000:3.0:1789762761.160179:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762761.160181:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067299328 00000100:00100000:2.0:1789762761.160198:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 317 00000100:00100000:2.0:1789762761.160211:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb93b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067299200:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762761.160219:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762761.160241:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067299456 00000100:00100000:0.0:1789762761.160245:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067299456 00000100:00100000:1.0:1789762761.160249:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067299584 00000100:00100000:1.0:1789762761.160256:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067299584 00000100:00100000:3.0:1789762761.160263:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762761.160279:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb93b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067299200:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 70us (201us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762761.160282:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067299712 00000100:00100000:0.0:1789762761.160287:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067299712 00000100:00100000:2.0:1789762761.160288:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 317 00000100:00100000:1.0:1789762761.160294:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 278 00000100:00100000:3.0:1789762761.160305:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 279 00000100:00100000:1.0:1789762761.160308:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d274c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2395:x1876702067299328:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762761.160321:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067299456:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762761.160327:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 280 00000100:00100000:0.0:1789762761.160346:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067299584:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762761.160359:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d274c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2395:x1876702067299328:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 56us (239us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762761.160366:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067299456:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 48us (186us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762761.160373:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 278 00000100:00100000:3.0:1789762761.160380:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 279 00000100:00100000:0.0:1789762761.160396:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067299584:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 57us (177us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762761.160397:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 281 00000100:00100000:1.0:1789762761.160413:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067299712:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762761.160414:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 280 00000100:00080000:3.0:1789762761.160428:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762761.160434:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762761.160446:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067299712:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 39us (182us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762761.160452:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 281 00000100:00100000:1.0:1789762761.279158:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234526107950 time 234526205035 state 0x0 ref 1 00000100:00100000:1.0:1789762761.279219:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067299072:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762761.279274:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b56 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.279297:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762761.279303:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762761.279311:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b56 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.279329:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b56 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.283985:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b56 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.284008:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b56 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.284021:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b56 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.284031:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762761.284038:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b56 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.284049:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b56 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.284088:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b56 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.284099:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b56 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762761.284215:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067299072:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5006us (200031us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762761.284227:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 214 00000100:00100000:1.0:1789762761.284236:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234526107950 time 234526205035 state 0x2 ref 0 00000100:00100000:3.0:1789762761.286969:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762761.287032:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067299840 00000100:00100000:0.0:1789762761.287046:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067299840 00000100:00100000:0.0:1789762761.287097:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234526107950 time 234526205035 state 0x0 ref 1 00000100:00100000:0.0:1789762761.287112:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234726205035 time 234526205035 state 0x1 ref 1 00010000:00010000:2.0:1789762761.344066:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762761.479235:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234726205035 time 234726306607 state 0x0 ref 1 00000100:00100000:1.0:1789762761.479263:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067299840:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762761.479303:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b5d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.479318:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762761.479322:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762761.479327:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b5d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.479339:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b5d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.483435:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b5d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.483447:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b5d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.483455:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b5d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.483462:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762761.483466:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b5d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.483473:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b5d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.483489:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b5d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762761.483495:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b5d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762761.483551:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067299840:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4300us (196631us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762761.483558:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 215 00000100:00100000:1.0:1789762761.483570:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234726205035 time 234726306607 state 0x2 ref 0 00000100:00100000:3.0:1789762761.485662:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762761.485738:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067299968 00000100:00100000:0.0:1789762761.485749:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067299968 00000100:00100000:0.0:1789762761.485816:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234726205035 time 234726306607 state 0x0 ref 1 00000100:00100000:0.0:1789762761.485828:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234926306607 time 234726306607 state 0x1 ref 1 00000100:00100000:0.0:1789762761.679325:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234926306607 time 234926394270 state 0x0 ref 1 00000100:00100000:0.0:1789762761.679362:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067299968:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762761.679430:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f174b64 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.679468:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762761.679486:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762761.679492:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f174b64 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.679504:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f174b64 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.683244:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f174b64 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.683260:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f174b64 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.683270:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f174b64 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.683279:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762761.683283:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f174b64 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.683292:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f174b64 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.683311:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f174b64 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762761.683318:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f976f348/0xdf5a7ac59f174b64 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762761.683422:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067299968:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4076us (197759us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762761.683436:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 216 00000100:00100000:0.0:1789762761.683444:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234926306607 time 234926394270 state 0x2 ref 0 00000100:00100000:2.0:1789762761.686056:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762761.686118:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067300096 00000100:00100000:1.0:1789762761.686128:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067300096 00000100:00100000:1.0:1789762761.686182:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 234926306607 time 234926394270 state 0x0 ref 1 00000100:00100000:1.0:1789762761.686193:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235126394270 time 234926394270 state 0x1 ref 1 00000100:00100000:3.0:1789762761.879501:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235126394270 time 235126559925 state 0x0 ref 1 00000100:00100000:3.0:1789762761.879554:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067300096:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762761.879638:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d5b57141/0xdf5a7ac59f174b6b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762761.879686:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762761.879694:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762761.879706:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5b57141/0xdf5a7ac59f174b6b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762761.879727:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5b57141/0xdf5a7ac59f174b6b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762761.885452:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d5b57141/0xdf5a7ac59f174b6b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762761.885474:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5b57141/0xdf5a7ac59f174b6b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762761.885488:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5b57141/0xdf5a7ac59f174b6b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762761.885499:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d999dc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762761.885506:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5b57141/0xdf5a7ac59f174b6b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762761.885517:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5b57141/0xdf5a7ac59f174b6b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762761.885543:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5b57141/0xdf5a7ac59f174b6b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762761.885574:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5b57141/0xdf5a7ac59f174b6b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762761.885668:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067300096:12345-192.168.203.37@tcp:3:dd.500 Request processed in 6138us (199611us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762761.885679:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 217 00000100:00100000:3.0:1789762761.885686:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235126394270 time 235126559925 state 0x2 ref 0 00000100:00100000:3.0:1789762761.889011:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762761.889122:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067300224 00000100:00100000:1.0:1789762761.889162:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067300224 00000100:00100000:1.0:1789762761.889217:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235126394270 time 235126559925 state 0x0 ref 1 00000100:00100000:1.0:1789762761.889232:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235326559925 time 235126559925 state 0x1 ref 1 00000100:00100000:1.0:1789762762.079656:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235326559925 time 235326722972 state 0x0 ref 1 00000100:00100000:1.0:1789762762.079693:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067300224:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762762.079764:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b72 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.079790:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762762.079798:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762762.079810:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b72 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.079837:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b72 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.084446:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b72 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.084495:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b72 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.084508:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b72 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.084517:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762762.084523:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b72 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.084535:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b72 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.084557:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b72 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.084567:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b72 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762762.084659:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067300224:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4980us (195650us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762762.084673:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 218 00000100:00100000:1.0:1789762762.084681:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235326559925 time 235326722972 state 0x2 ref 0 00000100:00100000:3.0:1789762762.088014:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762762.088099:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067300352 00000100:00100000:0.0:1789762762.088114:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067300352 00000100:00100000:0.0:1789762762.088196:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235326559925 time 235326722972 state 0x0 ref 1 00000100:00100000:0.0:1789762762.088212:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235526722972 time 235326722972 state 0x1 ref 1 00000100:00100000:1.0:1789762762.240206:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074632576:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762762.240264:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff70c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074632832:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762762.240331:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762762.240348:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074632448:0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.240418:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074632576 00000100:00100000:3.0:1789762762.240431:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074632576 00000100:00100000:2.0:1789762762.240442:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762762.240471:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074633088:0@lo:400:kworker.0 00000100:00100000:1.0:1789762762.240534:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762762.240542:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 118 00000100:00100000:2.0:1789762762.240550:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074632960:0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.240567:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074632576:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762762.240586:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074633600:0@lo:400:kworker.0 00002000:00000024:3.0:1789762762.240639:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762762.240669:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762762.240671:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fdfe1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074632704:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762762.240747:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074632576:12345-0@lo:13:osp-pre-0-1.0 Request processed in 187us (417us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762762.240748:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074633472:0@lo:400:kworker.0 00000100:00100000:0.0:1789762762.240762:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074633088 00000100:00100000:1.0:1789762762.240767:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762762.240774:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074633088 00000100:00100000:3.0:1789762762.240777:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 118 00000100:00100000:1.0:1789762762.240813:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074633216:0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.240824:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074632704 00000100:00100000:2.0:1789762762.240826:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762762.240832:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074632704 00000100:00100000:0.0:1789762762.240846:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1789762762.240868:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074633088:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762762.240868:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762762.240884:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074633984:0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.240899:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 119 00000100:00100000:3.0:1789762762.240951:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074632704:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762762.240959:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074633728:0@lo:400:kworker.0 00002000:00000024:3.0:1789762762.240971:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762762.241008:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762762.241035:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762762.241036:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074633088:12345-0@lo:400:kworker.0 Request processed in 172us (501us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762762.241050:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074632704:12345-0@lo:13:osp-pre-0-0.0 Request processed in 138us (293us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762762.241053:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:2.0:1789762762.241073:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1789762762.241083:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074634240:0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.241092:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 119 00000100:00100000:2.0:1789762762.241095:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074633856:0@lo:400:kworker.0 00000100:00100000:1.0:1789762762.241155:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762762.241163:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074632448 00000100:00100000:3.0:1789762762.241171:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074632448 00000100:00100000:2.0:1789762762.241182:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762762.241233:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 318 00000100:00100000:0.0:1789762762.241235:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074632960 00000100:00100000:1.0:1789762762.241237:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fdfe1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074632704:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762762.241240:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074634368:0@lo:400:kworker.0 00000100:00100000:0.0:1789762762.241246:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074632960 00000100:00100000:3.0:1789762762.241255:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb939850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074632448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762762.241313:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074633472 00000100:00100000:2.0:1789762762.241316:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762762.241320:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074633472 00000100:00100000:1.0:1789762762.241329:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074633216 00000100:00100000:1.0:1789762762.241341:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074633216 00000100:00100000:3.0:1789762762.241355:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb939850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074632448:12345-0@lo:400:kworker.0 Request processed in 99us (912us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762762.241356:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074633728 00000100:00100000:0.0:1789762762.241365:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074633728 00000100:00100000:3.0:1789762762.241375:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 318 00000100:00100000:2.0:1789762762.241387:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074632448:0@lo:400:kworker.0 00000100:00100000:1.0:1789762762.241399:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074633856 00000100:00100000:1.0:1789762762.241408:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074633856 00000100:00100000:0.0:1789762762.241419:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:0.0:1789762762.241439:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074632960:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789762762.241439:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762762.241451:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762762.241458:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283 00000100:00080000:3.0:1789762762.241463:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762762.241478:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074633472:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.241499:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:3.0:1789762762.241518:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074633216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762762.241525:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:0.0:1789762762.241540:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074632960:12345-0@lo:400:kworker.0 Request processed in 104us (873us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762762.241551:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074633728:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762762.241557:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074633472:12345-0@lo:400:kworker.0 Request processed in 84us (733us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762762.241576:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283 00000100:00100000:0.0:1789762762.241610:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00100000:3.0:1789762762.241617:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074633216:12345-0@lo:400:kworker.0 Request processed in 88us (737us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762762.241629:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:3.0:1789762762.241646:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00100000:2.0:1789762762.241650:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074633728:12345-0@lo:400:kworker.0 Request processed in 103us (614us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762762.241651:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074633856:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762762.241677:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00000100:00100000:0.0:1789762762.241687:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762762.241732:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074633216:0@lo:400:kworker.0 00000100:00100000:2.0:1789762762.241740:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074633984 00000100:00100000:2.0:1789762762.241750:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074633984 00000100:00100000:0.0:1789762762.241760:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074633344:0@lo:400:kworker.0 00000100:00100000:1.0:1789762762.241762:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074633856:12345-0@lo:400:kworker.0 Request processed in 115us (577us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762762.241767:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074633728:0@lo:400:kworker.0 00000100:00100000:1.0:1789762762.241792:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:2.0:1789762762.241808:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074634240 00000100:00100000:2.0:1789762762.241816:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074634240 00000100:00100000:3.0:1789762762.241833:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074634368 00000100:00100000:3.0:1789762762.241842:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074634368 00000100:00100000:0.0:1789762762.241848:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762762.241867:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 112 00000100:00100000:2.0:1789762762.241880:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 113 00000100:00100000:1.0:1789762762.241889:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074633984:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762762.241908:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762762.241909:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074634240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.241944:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 114 00000100:00100000:3.0:1789762762.241965:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074634368:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762762.241980:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff881f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074634496:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762762.241985:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074633984:12345-0@lo:400:kworker.0 Request processed in 104us (979us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762762.242000:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 112 00000100:00100000:1.0:1789762762.242041:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762762.242060:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074634368:12345-0@lo:400:kworker.0 Request processed in 101us (748us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762762.242075:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074634240:12345-0@lo:400:kworker.0 Request processed in 168us (936us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762762.242083:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762762.242083:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 114 00000100:00100000:1.0:1789762762.242084:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074634112:0@lo:400:kworker.0 00000100:00100000:2.0:1789762762.242107:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 113 00000100:00080000:3.0:1789762762.242158:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762762.242185:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762762.242207:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074634112 00000100:00100000:2.0:1789762762.242220:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074634112 00000100:00100000:0.0:1789762762.242238:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074633344 00000100:00100000:3.0:1789762762.242243:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074634240:0@lo:400:kworker.0 00000100:00100000:0.0:1789762762.242251:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074633344 00000100:00100000:1.0:1789762762.242279:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074632576:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1789762762.242297:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762762.242341:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 115 00000100:00100000:0.0:1789762762.242349:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:1.0:1789762762.242365:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074633088:0@lo:400:kworker.0 00000100:00100000:2.0:1789762762.242369:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074634112:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762762.242370:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074633344:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.242427:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074633856:0@lo:400:kworker.0 00000100:00100000:1.0:1789762762.242451:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074633600 00000100:00100000:3.0:1789762762.242457:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074634368:0@lo:400:kworker.0 00000100:00100000:1.0:1789762762.242460:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074633600 00000100:00100000:2.0:1789762762.242487:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074634112:12345-0@lo:400:kworker.0 Request processed in 126us (321us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762762.242493:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074632960:0@lo:400:kworker.0 00000100:00100000:0.0:1789762762.242509:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074633344:12345-0@lo:400:kworker.0 Request processed in 142us (662us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762762.242510:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 115 00000100:00100000:1.0:1789762762.242520:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:3.0:1789762762.242531:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074633472:0@lo:400:kworker.0 00000100:00100000:0.0:1789762762.242542:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00100000:1.0:1789762762.242544:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074633600:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.242556:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074633984:0@lo:400:kworker.0 00000100:00080000:2.0:1789762762.242589:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762762.242607:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074632832 00000100:00100000:0.0:1789762762.242615:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074632832 00000100:00100000:1.0:1789762762.242623:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074633600:12345-0@lo:400:kworker.0 Request processed in 82us (578us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762762.242639:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00100000:2.0:1789762762.242642:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074634496 00000100:00100000:2.0:1789762762.242669:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074634496 00000100:00100000:0.0:1789762762.242680:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 120 00000100:00100000:0.0:1789762762.242697:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074632832:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762762.242719:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074633600:0@lo:400:kworker.0 00002000:00000024:0.0:1789762762.242726:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762762.242734:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 121 00000100:00100000:1.0:1789762762.242746:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074634112:0@lo:400:kworker.0 00000100:00100000:2.0:1789762762.242757:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074634496:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789762762.242781:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762762.242800:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074632832:12345-0@lo:13:osp-pre-1-0.0 Request processed in 108us (1118us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762762.242814:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 120 00000100:00100000:3.0:1789762762.242815:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff70c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074632832:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762762.242847:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074633344:0@lo:400:kworker.0 00000100:00100000:2.0:1789762762.242854:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074634496:12345-0@lo:13:osp-pre-1-1.0 Request processed in 103us (785us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762762.242870:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 121 00000100:00100000:3.0:1789762762.242891:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff881f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074634496:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762762.279759:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235526722972 time 235526827242 state 0x0 ref 1 00000100:00100000:0.0:1789762762.279794:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067300352:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762762.279842:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b79 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762762.279881:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762762.279887:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762762.279896:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b79 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762762.279946:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b79 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762762.284229:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b79 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762762.284248:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b79 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762762.284260:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b79 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762762.284269:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762762.284275:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b79 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762762.284303:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b79 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762762.284325:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b79 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762762.284334:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b79 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762762.284407:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067300352:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4626us (196407us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762762.284417:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 219 00000100:00100000:3.0:1789762762.284427:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235526722972 time 235526827242 state 0x2 ref 0 00000100:00100000:0.0:1789762762.287543:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762762.287608:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067300480 00000100:00100000:3.0:1789762762.287622:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067300480 00000100:00100000:3.0:1789762762.287675:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235526722972 time 235526827242 state 0x0 ref 1 00000100:00100000:3.0:1789762762.287689:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235726827242 time 235526827242 state 0x1 ref 1 00010000:00010000:1.0:1789762762.368172:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762762.480543:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235726827242 time 235727602656 state 0x0 ref 1 00000100:00100000:0.0:1789762762.480595:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067300480:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762762.480674:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b80 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762762.480700:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762762.480705:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762762.480714:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b80 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762762.480733:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b80 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.486269:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b80 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.486320:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b80 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.486337:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b80 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.486351:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762762.486359:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b80 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.486375:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b80 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.486406:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b80 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.486421:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5e0e71b/0xdf5a7ac59f174b80 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762762.486560:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067300480:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5981us (199014us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762762.486581:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 220 00000100:00100000:1.0:1789762762.486594:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235726827242 time 235727602656 state 0x2 ref 0 00000100:00100000:2.0:1789762762.490834:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762762.490891:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067300608 00000100:00100000:1.0:1789762762.490961:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067300608 00000100:00100000:1.0:1789762762.491016:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235726827242 time 235727602656 state 0x0 ref 1 00000100:00100000:1.0:1789762762.491032:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235927602656 time 235727602656 state 0x1 ref 1 00000100:00100000:1.0:1789762762.680608:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235927602656 time 235927677867 state 0x0 ref 1 00000100:00100000:1.0:1789762762.680635:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067300608:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762762.680674:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b87 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.680688:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762762.680692:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762762.680697:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b87 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.680708:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b87 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.683731:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b87 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.683742:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b87 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.683749:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b87 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.683769:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762762.683773:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b87 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.683780:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b87 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.683793:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b87 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.683798:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b87 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762762.683849:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067300608:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3226us (193019us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762762.683857:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 221 00000100:00100000:1.0:1789762762.683863:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235927602656 time 235927677867 state 0x2 ref 0 00000100:00100000:0.0:1789762762.686053:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762762.686102:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067300736 00000100:00100000:3.0:1789762762.686111:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067300736 00000100:00100000:3.0:1789762762.686153:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 235927602656 time 235927677867 state 0x0 ref 1 00000100:00100000:3.0:1789762762.686162:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236127677867 time 235927677867 state 0x1 ref 1 00000100:00100000:1.0:1789762762.880740:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236127677867 time 236127811184 state 0x0 ref 1 00000100:00100000:1.0:1789762762.880768:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067300736:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762762.880806:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b8e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.880819:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762762.880836:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762762.880842:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b8e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.880856:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b8e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.883508:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b8e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.883519:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b8e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.883525:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b8e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.883530:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762762.883534:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b8e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.883541:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b8e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.883554:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b8e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762762.883558:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174b8e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762762.883621:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067300736:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2866us (197575us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762762.883630:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 222 00000100:00100000:1.0:1789762762.883636:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236127677867 time 236127811184 state 0x2 ref 0 00000100:00100000:2.0:1789762762.885726:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762762.885766:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067300864 00000100:00100000:1.0:1789762762.885774:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067300864 00000100:00100000:1.0:1789762762.885804:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236127677867 time 236127811184 state 0x0 ref 1 00000100:00100000:1.0:1789762762.885813:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236327811184 time 236127811184 state 0x1 ref 1 00000100:00100000:0.0:1789762763.080814:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236327811184 time 236327887004 state 0x0 ref 1 00000100:00100000:0.0:1789762763.080863:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff883800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067300864:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762763.080905:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b95 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789762763.080942:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762763.080952:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762763.080958:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762763.080963:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b95 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.080975:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b95 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.083617:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b95 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.083632:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b95 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.083643:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b95 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.083652:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c992ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762763.083657:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b95 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.083668:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b95 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.083695:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b95 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.083702:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b95 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762763.083750:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff883800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067300864:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2907us (198025us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762763.083765:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 223 00000100:00100000:0.0:1789762763.083770:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236327811184 time 236327887004 state 0x2 ref 0 00000100:00100000:0.0:1789762763.085692:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762763.085738:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067300992 00000100:00100000:1.0:1789762763.085746:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067300992 00000100:00100000:1.0:1789762763.085777:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236327811184 time 236327887004 state 0x0 ref 1 00000100:00100000:1.0:1789762763.085786:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236527887004 time 236327887004 state 0x1 ref 1 00000100:00100000:0.0:1789762763.280899:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236527887004 time 236527970776 state 0x0 ref 1 00000100:00100000:0.0:1789762763.280939:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067300992:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762763.280973:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b9c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.280987:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762763.280991:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762763.280996:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b9c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.281007:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b9c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.283902:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b9c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.283937:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b9c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.283944:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b9c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.283949:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c992ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762763.283952:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b9c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.283958:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b9c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.283974:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b9c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.283983:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002263d34e/0xdf5a7ac59f174b9c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762763.284042:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067300992:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3124us (198346us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762763.284051:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 224 00000100:00100000:3.0:1789762763.284058:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236527887004 time 236527970776 state 0x2 ref 0 00000100:00100000:3.0:1789762763.285840:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762763.285902:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067301120 00000100:00100000:1.0:1789762763.285911:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067301120 00000100:00100000:1.0:1789762763.285974:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236527887004 time 236527970776 state 0x0 ref 1 00000100:00100000:1.0:1789762763.285984:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236727970776 time 236527970776 state 0x1 ref 1 00010000:00010000:0.0:1789762763.392089:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762763.481005:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236727970776 time 236728078112 state 0x0 ref 1 00000100:00100000:0.0:1789762763.481035:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067301120:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762763.481074:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174ba3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.481088:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762763.481092:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762763.481097:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174ba3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.481108:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174ba3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.484079:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174ba3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.484090:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174ba3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.484096:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174ba3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.484102:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c992e400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762763.484112:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174ba3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.484130:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174ba3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.484145:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174ba3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.484151:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174ba3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762763.484196:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067301120:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3176us (198356us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762763.484204:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 225 00000100:00100000:0.0:1789762763.484209:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236727970776 time 236728078112 state 0x2 ref 0 00000100:00100000:3.0:1789762763.486016:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762763.486048:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067301248 00000100:00100000:0.0:1789762763.486055:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067301248 00000100:00100000:0.0:1789762763.486086:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236727970776 time 236728078112 state 0x0 ref 1 00000100:00100000:0.0:1789762763.486094:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236928078112 time 236728078112 state 0x1 ref 1 00000100:00100000:3.0:1789762763.681173:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236928078112 time 236928244379 state 0x0 ref 1 00000100:00100000:3.0:1789762763.681202:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067301248:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762763.681240:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174baa lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.681254:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762763.681257:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762763.681262:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174baa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.681273:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174baa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.684076:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174baa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.684089:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174baa lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.684130:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174baa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.684139:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762763.684144:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174baa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.684154:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174baa lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.684172:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174baa lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.684180:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f174baa lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762763.684242:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067301248:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3053us (198227us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762763.684252:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 226 00000100:00100000:0.0:1789762763.684260:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236928078112 time 236928244379 state 0x2 ref 0 00000100:00100000:2.0:1789762763.686421:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762763.686461:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067301376 00000100:00100000:3.0:1789762763.686469:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067301376 00000100:00100000:3.0:1789762763.686499:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 236928078112 time 236928244379 state 0x0 ref 1 00000100:00100000:3.0:1789762763.686508:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237128244379 time 236928244379 state 0x1 ref 1 00000100:00100000:0.0:1789762763.881270:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237128244379 time 237128342721 state 0x0 ref 1 00000100:00100000:0.0:1789762763.881299:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067301376:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762763.881336:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174bb1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.881364:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762763.881367:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762763.881372:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174bb1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.881386:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174bb1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762763.884280:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174bb1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762763.884291:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174bb1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762763.884309:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174bb1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762763.884314:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c992e400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762763.884319:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174bb1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762763.884325:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174bb1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762763.884339:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174bb1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762763.884345:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a5d95ff/0xdf5a7ac59f174bb1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762763.884391:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067301376:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3105us (197971us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762763.884398:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 227 00000100:00100000:1.0:1789762763.884404:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237128244379 time 237128342721 state 0x2 ref 0 00000100:00100000:2.0:1789762763.886432:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762763.886509:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067301504 00000100:00100000:0.0:1789762763.886518:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067301504 00000100:00100000:0.0:1789762763.886549:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237128244379 time 237128342721 state 0x0 ref 1 00000100:00100000:0.0:1789762763.886557:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237328342721 time 237128342721 state 0x1 ref 1 00000100:00100000:1.0:1789762764.081391:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237328342721 time 237328460198 state 0x0 ref 1 00000100:00100000:1.0:1789762764.081422:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff883100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067301504:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762764.081467:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bb8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.081501:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762764.081506:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762764.081513:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bb8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.081528:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bb8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.085332:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bb8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.085347:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bb8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.085356:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bb8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.085363:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762764.085368:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bb8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.085377:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bb8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.085394:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bb8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.085417:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bb8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762764.085490:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff883100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067301504:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4077us (199055us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762764.085504:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 228 00000100:00100000:1.0:1789762764.085513:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237328342721 time 237328460198 state 0x2 ref 0 00000100:00100000:3.0:1789762764.088590:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762764.088655:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067301632 00000100:00100000:0.0:1789762764.088665:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067301632 00000100:00100000:0.0:1789762764.088715:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237328342721 time 237328460198 state 0x0 ref 1 00000100:00100000:0.0:1789762764.088732:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237528460198 time 237328460198 state 0x1 ref 1 00000100:00100000:1.0:1789762764.281521:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237528460198 time 237528589663 state 0x0 ref 1 00000100:00100000:1.0:1789762764.281560:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067301632:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762764.281613:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bbf lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.281633:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762764.281638:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762764.281645:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bbf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.281662:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bbf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.285959:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bbf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.286003:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bbf lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.286015:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bbf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.286022:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762764.286026:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bbf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.286035:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bbf lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.286053:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bbf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762764.286060:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f174bbf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762764.286142:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067301632:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4598us (197552us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762764.286154:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 229 00000100:00100000:1.0:1789762764.286162:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237528460198 time 237528589663 state 0x2 ref 0 00000100:00100000:3.0:1789762764.288102:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789762764.288139:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fdfe3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074634624:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762764.288198:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762764.288297:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074634624 00000100:00100000:1.0:1789762764.288315:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074634624 00000100:00100000:1.0:1789762764.288386:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:1.0:1789762764.288400:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074634624:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762764.288439:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762764.288507:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074634624:12345-0@lo:41:osp-pre-1-0.0 Request processed in 109us (309us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762764.288519:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:3.0:1789762764.288671:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fdfe3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074634624:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762764.288723:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762764.288770:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067301760 00000100:00100000:1.0:1789762764.288777:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067301760 00000100:00100000:1.0:1789762764.288805:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237528460198 time 237528589663 state 0x0 ref 1 00000100:00100000:1.0:1789762764.288815:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237728589663 time 237528589663 state 0x1 ref 1 00010000:00010000:2.0:1789762764.416102:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762764.481720:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237728589663 time 237728787329 state 0x0 ref 1 00000100:00100000:2.0:1789762764.481764:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067301760:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762764.481820:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c59f8506/0xdf5a7ac59f174bc6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762764.481869:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762764.481875:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762764.481884:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c59f8506/0xdf5a7ac59f174bc6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762764.481901:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c59f8506/0xdf5a7ac59f174bc6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.486502:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c59f8506/0xdf5a7ac59f174bc6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.486523:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c59f8506/0xdf5a7ac59f174bc6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.486536:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c59f8506/0xdf5a7ac59f174bc6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.486558:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7fc7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762764.486566:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c59f8506/0xdf5a7ac59f174bc6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.486578:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c59f8506/0xdf5a7ac59f174bc6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.486617:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c59f8506/0xdf5a7ac59f174bc6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.486633:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c59f8506/0xdf5a7ac59f174bc6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762764.486745:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067301760:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4999us (198020us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762764.486766:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 230 00000100:00100000:0.0:1789762764.486778:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237728589663 time 237728787329 state 0x2 ref 0 00000100:00100000:0.0:1789762764.490031:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762764.490182:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067301888 00000100:00100000:1.0:1789762764.490197:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067301888 00000100:00100000:1.0:1789762764.490250:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237728589663 time 237728787329 state 0x0 ref 1 00000100:00100000:1.0:1789762764.490266:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237928787329 time 237728787329 state 0x1 ref 1 00000100:00100000:3.0:1789762764.681849:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237928787329 time 237928918808 state 0x0 ref 1 00000100:00100000:3.0:1789762764.681909:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067301888:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762764.681987:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a8d5b026/0xdf5a7ac59f174bcd lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.682007:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762764.682012:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762764.682019:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8d5b026/0xdf5a7ac59f174bcd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.682036:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8d5b026/0xdf5a7ac59f174bcd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.685792:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a8d5b026/0xdf5a7ac59f174bcd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.685806:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8d5b026/0xdf5a7ac59f174bcd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.685816:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8d5b026/0xdf5a7ac59f174bcd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.685825:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee739a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762764.685830:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8d5b026/0xdf5a7ac59f174bcd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.685840:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8d5b026/0xdf5a7ac59f174bcd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.685870:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8d5b026/0xdf5a7ac59f174bcd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.685879:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8d5b026/0xdf5a7ac59f174bcd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762764.685986:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067301888:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4087us (195955us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762764.685997:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 231 00000100:00100000:3.0:1789762764.686005:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237928787329 time 237928918808 state 0x2 ref 0 00000100:00100000:2.0:1789762764.688516:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762764.688602:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067302016 00000100:00100000:1.0:1789762764.688616:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067302016 00000100:00100000:1.0:1789762764.688669:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 237928787329 time 237928918808 state 0x0 ref 1 00000100:00100000:1.0:1789762764.688686:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 238128918808 time 237928918808 state 0x1 ref 1 00000100:00100000:0.0:1789762764.882076:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 238128918808 time 238129144013 state 0x0 ref 1 00000100:00100000:0.0:1789762764.882150:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff883b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067302016:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762764.882218:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bd4 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.882276:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762764.882282:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762764.882291:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bd4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.882312:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bd4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.886948:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bd4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.886967:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bd4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.886979:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bd4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.886988:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762764.886994:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bd4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.887004:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bd4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.887023:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bd4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.887046:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bd4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762764.887154:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff883b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067302016:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5052us (198635us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762764.887170:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 232 00000100:00100000:0.0:1789762764.887181:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 238128918808 time 238129144013 state 0x2 ref 0 00000100:00100000:0.0:1789762764.890152:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762764.890274:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067302144 00000100:00100000:2.0:1789762764.890286:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067302144 00000100:00100000:2.0:1789762764.890336:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 238128918808 time 238129144013 state 0x0 ref 1 00000100:00100000:2.0:1789762764.890349:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 238329144013 time 238129144013 state 0x1 ref 1 00000100:00100000:0.0:1789762765.082211:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 238329144013 time 238329279819 state 0x0 ref 1 00000100:00100000:0.0:1789762765.082281:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067302144:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:2.0:1789762765.082310:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762765.082343:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bdb lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.082365:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762765.082371:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762765.082379:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bdb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.082396:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bdb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.086769:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bdb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.086795:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bdb lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.086814:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bdb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.086831:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762765.086841:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bdb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.086859:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bdb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.086910:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bdb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.086964:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b3ab64b/0xdf5a7ac59f174bdb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762765.087069:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067302144:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4828us (196916us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762765.087088:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 233 00000100:00100000:3.0:1789762765.087101:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 238329144013 time 238329279819 state 0x2 ref 0 00000100:00100000:3.0:1789762765.089960:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762765.090037:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067302272 00000100:00100000:0.0:1789762765.090048:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067302272 00000100:00100000:0.0:1789762765.090122:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 238329144013 time 238329279819 state 0x0 ref 1 00000100:00100000:0.0:1789762765.090138:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 238529279819 time 238329279819 state 0x1 ref 1 00000100:00100000:0.0:1789762765.282255:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 238529279819 time 238529327636 state 0x0 ref 1 00000100:00100000:0.0:1789762765.282302:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067302272:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762765.282344:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174be2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.282361:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762765.282366:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762765.282372:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174be2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.282385:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174be2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.285402:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174be2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.285432:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174be2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.285442:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174be2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.285450:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762765.285470:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174be2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.285482:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174be2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.285501:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174be2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.285509:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f174be2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762765.285591:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067302272:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3290us (195621us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762765.285605:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 234 00000100:00100000:0.0:1789762765.285614:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 238529279819 time 238529327636 state 0x2 ref 0 00000100:00100000:2.0:1789762765.287630:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762765.287695:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067302528 00000100:00100000:3.0:1789762765.287707:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067302528 00000100:00080000:2.0:1789762765.287802:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762765.287812:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 37 00000100:00100000:3.0:1789762765.287835:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:12196:x1876702067302528:12345-192.168.203.37@tcp:35:dd.500 00000100:00100000:3.0:1789762765.288201:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:12196:x1876702067302528:12345-192.168.203.37@tcp:35:dd.500 Request processed in 374us (574us total) trans 4294967358 rc 0/0 00000100:00100000:3.0:1789762765.288212:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 37 00000100:00100000:2.0:1789762765.312098:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762765.312145:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff881c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074634752:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762765.312225:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762765.312282:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074634752 00000100:00100000:0.0:1789762765.312292:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074634752 00000100:00100000:0.0:1789762765.312349:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:0.0:1789762765.312361:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074634752:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762765.312413:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762765.312475:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074634752:12345-0@lo:41:osp-pre-0-1.0 Request processed in 115us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762765.312486:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:2.0:1789762765.312546:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff881c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074634752:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762765.317030:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762765.317092:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067302656 00000100:00100000:1.0:1789762765.317099:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067302656 00000100:00100000:1.0:1789762765.317164:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 20 00000100:00100000:1.0:1789762765.317174:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c93850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:6823:x1876702067302656:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762765.317188:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762765.317240:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f174926 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c522 expref: 6 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.317265:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762765.317308:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c93850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:6823:x1876702067302656:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 136us (283us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762765.317316:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 20 00000100:00100000:0.0:1789762765.327772:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762765.327833:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067302784 00000100:00100000:3.0:1789762765.327842:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067302784 00000100:00100000:3.0:1789762765.327887:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 155 00000100:00100000:3.0:1789762765.327898:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bde680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:11101:x1876702067302784:12345-192.168.203.37@tcp:101:bash.0 00010000:00010000:3.0:1789762765.327943:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762765.327963:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174be9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c529 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.328050:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174bf0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.328064:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174bf0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.328076:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174bf0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762765.328128:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x9:0x0]: returning size 16384 00000004:00200000:3.0:1789762765.328132:0:11692:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762765.328153:0:11692:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174bf0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.328176:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174be9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c529 expref: 12 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.328190:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174bf0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c529 expref: 11 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.328198:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:3.0:1789762765.328202:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762765.328204:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7fc6600, rc 0) 00000100:00100000:3.0:1789762765.328246:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bde680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:11101:x1876702067302784:12345-192.168.203.37@tcp:101:bash.0 Request processed in 350us (477us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762765.328252:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 155 00000100:00100000:2.0:1789762765.333092:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762765.333170:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067303040 00000100:00100000:1.0:1789762765.333177:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067303040 00000100:00100000:1.0:1789762765.333223:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 156 00000100:00100000:1.0:1789762765.333237:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:12211:x1876702067303040:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:1.0:1789762765.333252:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762765.333267:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174bf7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c530 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762765.333335:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x9:0x0]: returning size 16384 00010000:00010000:1.0:1789762765.333475:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174bf7 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c530 expref: 13 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.333494:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762765.333499:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789762765.333505:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174bf7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c530 expref: 13 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.333520:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a95400, rc 0) 00000100:00100000:1.0:1789762765.333564:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:12211:x1876702067303040:12345-192.168.203.37@tcp:101:rm.0 Request processed in 331us (475us total) trans 4294967359 rc 301/301 00000100:00100000:1.0:1789762765.333620:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 156 00000100:00100000:2.0:1789762765.335413:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762765.335504:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067303168 00000100:00100000:1.0:1789762765.335518:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067303168 00000100:00100000:1.0:1789762765.335600:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 38 00000100:00100000:1.0:1789762765.335623:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:12211:x1876702067303168:12345-192.168.203.37@tcp:37:rm.0 00000100:00100000:1.0:1789762765.338738:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:12211:x1876702067303168:12345-192.168.203.37@tcp:37:rm.0 Request processed in 3121us (3324us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762765.338749:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 38 00000100:00100000:2.0:1789762765.340301:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762765.340361:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067303424 00000100:00100000:3.0:1789762765.340369:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067303424 00000100:00100000:3.0:1789762765.340415:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 39 00000100:00100000:3.0:1789762765.340425:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:12211:x1876702067303424:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:3.0:1789762765.340560:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:12211:x1876702067303424:12345-192.168.203.37@tcp:35:rm.0 Request processed in 135us (260us total) trans 4294967360 rc 0/0 00000100:00100000:3.0:1789762765.340569:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 39 00000100:00100000:2.0:1789762765.341762:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762765.341863:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067303680 00000100:00100000:0.0:1789762765.341885:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067303680 00000100:00100000:0.0:1789762765.341960:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 157 00000100:00100000:0.0:1789762765.341988:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:12211:x1876702067303680:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:0.0:1789762765.342001:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762765.342014:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c05 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c537 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.342078:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c0c lrc: 3/1,0 mode: --/CR res: [0x200000402:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.342090:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c0c lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.342099:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c0c lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762765.342130:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x9:0x0]: returning size 16384 00010000:00010000:0.0:1789762765.342217:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c05 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c537 expref: 16 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.342233:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c0c lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x9:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c537 expref: 15 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.342241:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:0.0:1789762765.342244:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1789762765.342246:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa6c00, rc 0) 00000100:00100000:0.0:1789762765.342287:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:12211:x1876702067303680:12345-192.168.203.37@tcp:101:rm.0 Request processed in 318us (527us total) trans 4294967361 rc 0/0 00000100:00100000:0.0:1789762765.342294:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 157 00000100:00100000:0.0:1789762765.343411:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762765.343462:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067303808 00000100:00100000:2.0:1789762765.343470:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067303808 00000100:00100000:2.0:1789762765.343513:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 158 00000100:00100000:2.0:1789762765.343525:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:12211:x1876702067303808:12345-192.168.203.37@tcp:41:rm.0 00000004:00000024:2.0:1789762765.343577:0:13389:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7614758912 avail 7610564608 00000100:00100000:2.0:1789762765.343650:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:12211:x1876702067303808:12345-192.168.203.37@tcp:41:rm.0 Request processed in 125us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762765.343660:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 158 00000100:00100000:2.0:1789762765.345075:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762765.345171:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067304064 00000100:00100000:3.0:1789762765.345181:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067304064 00000100:00100000:3.0:1789762765.345232:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 159 00000100:00100000:3.0:1789762765.345248:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:12211:x1876702067304064:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:3.0:1789762765.345282:0:6418:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1789762765.345300:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174918 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c51b expref: 15 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.345320:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935200/0xdf5a7ac59f1745c9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4f8 expref: 14 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.345330:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37db743c00) 00010000:00010000:3.0:1789762765.345342:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174bf0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c529 expref: 13 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.345350:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:3.0:1789762765.345372:0:6418:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762765.345394:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c1a lrc: 3/0,1 mode: --/CW res: [0x200000402:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.345414:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c1a lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.345458:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c1a lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.345497:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7800/0xdf5a7ac59f174c21 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x9:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.345511:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7800/0xdf5a7ac59f174c21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.345521:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7800/0xdf5a7ac59f174c21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.345932:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c28 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.345944:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c28 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.345953:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c28 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789762765.347526:0:6418:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].4, 64 off8448 00000004:00080000:3.0:1789762765.347541:0:6418:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 238594621670ns, next at 239594619297ns, rc = 0 00010000:00010000:3.0:1789762765.347625:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6000/0xdf5a7ac59f174c2f lrc: 3/0,1 mode: --/PW res: [0x200000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347662:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6000/0xdf5a7ac59f174c2f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347674:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6000/0xdf5a7ac59f174c2f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347701:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6000/0xdf5a7ac59f174c2f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347708:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6000/0xdf5a7ac59f174c2f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347714:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6000/0xdf5a7ac59f174c2f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347733:0:6418:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762765.347746:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c28 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347752:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c28 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347759:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c28 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347766:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7fc7e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762765.347769:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762765.347771:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c28 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762765.347785:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6000/0xdf5a7ac59f174c2f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347785:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c28 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347809:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37c72bad00) 00010000:00010000:3.0:1789762765.347813:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789762765.347817:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c28 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762765.347823:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7fc6000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762765.347832:0:6408:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6000/0xdf5a7ac59f174c2f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347837:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c28 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762765.347845:0:6408:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6000/0xdf5a7ac59f174c2f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xb:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347857:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c1a lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762765.347862:0:6408:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff9c37c72bad00) 00010000:00010000:3.0:1789762765.347867:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c1a lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762765.347868:0:6408:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789762765.347876:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6000/0xdf5a7ac59f174c2f lrc: 1/0,0 mode: --/PW res: [0x200000402:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347880:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c1a lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762765.347889:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6000/0xdf5a7ac59f174c2f lrc: 0/0,0 mode: --/PW res: [0x200000402:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347892:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c1a lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347907:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7fc7200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762765.347911:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c1a lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347939:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c1a lrc: 5/0,0 mode: CW/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762765.347942:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762765.347948:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762765.347954:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:3.0:1789762765.347957:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762765.347963:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762765.347964:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c1a lrc: 3/0,0 mode: --/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.347972:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c1a lrc: 0/0,0 mode: --/CW res: [0x200000402:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762765.347983:0:6418:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009bde3a10 reply state 00000000a3717c86 transno 4294967362 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7800/0xdf5a7ac59f174c21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.348002:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7800/0xdf5a7ac59f174c21 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.348008:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x4009f790 (ffff9c37c72ba100) 00010000:00010000:3.0:1789762765.348011:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1789762765.348013:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762765.348060:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:12211:x1876702067304064:12345-192.168.203.37@tcp:36:rm.0 Request processed in 2817us (2986us total) trans 4294967362 rc 0/0 00000100:00100000:3.0:1789762765.348068:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 159 00000100:00100000:0.0:1789762765.349402:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762765.349483:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067304320 00000100:00100000:1.0:1789762765.349490:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067304320 00000100:00100000:1.0:1789762765.349533:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 40 00000100:00100000:1.0:1789762765.349546:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbc850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:12211:x1876702067304320:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:1.0:1789762765.349709:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbc850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:12211:x1876702067304320:12345-192.168.203.37@tcp:35:rm.0 Request processed in 162us (308us total) trans 4294967363 rc 0/0 00000100:00100000:1.0:1789762765.349720:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 40 00000100:00100000:0.0:1789762765.350562:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762765.350660:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067304576 00000100:00100000:1.0:1789762765.350669:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067304576 00000100:00100000:1.0:1789762765.350711:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 160 00000100:00100000:1.0:1789762765.350721:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:12211:x1876702067304576:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:1.0:1789762765.350751:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1789762765.350766:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c0c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x9:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c537 expref: 15 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.350797:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174598 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x9:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4ea expref: 14 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.350827:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743200) 00010000:00010000:1.0:1789762765.350840:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174536 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c4d5 expref: 13 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.350849:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762765.350852:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762765.350863:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174c36 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.350872:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174c36 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.350879:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174c36 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.350890:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174c3d lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.350896:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174c3d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.350902:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174c3d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351251:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f174c44 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351261:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f174c44 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351280:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f174c44 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762765.351819:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x9:0x0]: returning size 16384 00010000:00010000:1.0:1789762765.351830:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f174c44 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351837:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f174c44 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351843:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f174c44 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351848:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762765.351851:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f174c44 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351857:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f174c44 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351865:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x0 (ffff9c37db743500) 00010000:00010000:1.0:1789762765.351868:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762765.351871:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f174c44 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351876:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f174c44 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351891:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174c36 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351896:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174c36 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351909:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174c36 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351916:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174c36 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351922:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762765.351927:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174c36 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351937:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174c36 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351979:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762765.351984:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762765.351989:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174c36 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.351997:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f174c36 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762765.352008:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000aa52247b reply state 00000000bf177d86 transno 4294967364 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174c3d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.352017:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174c3d lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762765.352023:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff9c37db743e00) 00010000:00010000:1.0:1789762765.352025:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1789762765.352027:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762765.352074:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:12211:x1876702067304576:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1354us (1514us total) trans 4294967364 rc 0/0 00000100:00100000:1.0:1789762765.352089:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 160 00010000:00010000:1.0:1789762765.440067:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762766.280043:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762766.280124:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067304704 00000100:00100000:2.0:1789762766.280151:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762766.280152:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067304704 00000100:00100000:3.0:1789762766.280219:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067304832 00000100:00100000:2.0:1789762766.280223:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762766.280230:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067304832 00000100:00100000:1.0:1789762766.280261:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 319 00000100:00100000:1.0:1789762766.280283:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb938c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067304704:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762766.280299:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762766.280310:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 288 00000100:00100000:3.0:1789762766.280334:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:2396:x1876702067304832:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762766.280367:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067304960 00000100:00100000:0.0:1789762766.280379:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067304960 00000100:00100000:1.0:1789762766.280388:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb938c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067304704:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 108us (349us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762766.280405:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 319 00000100:00100000:2.0:1789762766.280413:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762766.280426:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:2396:x1876702067304832:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 98us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762766.280440:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 288 00000100:00100000:0.0:1789762766.280456:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067305216 00000100:00100000:0.0:1789762766.280466:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067305216 00000100:00100000:3.0:1789762766.280483:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 289 00000100:00100000:3.0:1789762766.280503:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067304960:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762766.280532:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067305088 00000100:00100000:1.0:1789762766.280541:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067305088 00000100:00100000:0.0:1789762766.280548:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 290 00000100:00100000:0.0:1789762766.280570:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067305216:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762766.280577:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762766.280578:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067304960:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 74us (278us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762766.280591:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 289 00000100:00100000:1.0:1789762766.280621:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 116 00000100:00100000:3.0:1789762766.280634:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067305344 00000100:00100000:1.0:1789762766.280639:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067305088:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762766.280642:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067305344 00000100:00100000:0.0:1789762766.280664:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067305216:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 98us (261us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762766.280680:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 290 00000100:00100000:3.0:1789762766.280690:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 291 00000100:00100000:1.0:1789762766.280699:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067305088:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 63us (493us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762766.280707:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067305344:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762766.280727:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 116 00000100:00080000:2.0:1789762766.280734:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762766.280743:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762766.280784:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067305344:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 83us (215us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762766.280796:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 291 00000001:00080000:3.0:1789762766.356721:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967361 is committed 00000001:00080000:3.0:1789762766.356760:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967364 is committed 00000004:00080000:3.0:1789762766.356791:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 5 committed 00010000:00010000:1.0:1789762766.356798:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7800/0xdf5a7ac59f174c21 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.356819:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7800/0xdf5a7ac59f174c21 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.356831:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7800/0xdf5a7ac59f174c21 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.356841:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7fc7800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762766.356847:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7800/0xdf5a7ac59f174c21 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.356871:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7800/0xdf5a7ac59f174c21 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.356900:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x9:0x0].0x4009f790 (ffff9c37c72ba100) 00010000:00010000:1.0:1789762766.356905:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762766.356911:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7800/0xdf5a7ac59f174c21 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.356954:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7800/0xdf5a7ac59f174c21 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.356988:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174c3d lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.357002:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174c3d lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.357010:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174c3d lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.357018:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762766.357023:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174c3d lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.357031:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174c3d lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.357046:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff9c37db743e00) 00010000:00010000:1.0:1789762766.357051:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762766.357055:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174c3d lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762766.357060:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074634880:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1789762766.357089:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174c3d lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762766.357212:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762766.357261:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074634880 00000100:00100000:1.0:1789762766.357274:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074634880 00000100:00080000:2.0:1789762766.357334:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762766.357374:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 101 00000100:00100000:1.0:1789762766.357393:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074634880:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1789762766.357429:0:9038:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:4 count 1 00002000:00010000:1.0:1789762766.357468:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x4:0x0] initial LVB size: 104857600, mtime: 0x6aad9cb8, atime: 0x6aad9ca5, ctime: 0x6aad9cb8, blocks: 0x3200d 00010000:00010000:1.0:1789762766.357478:0:9038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174c4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.357497:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=239s v=(0 0) 00010000:00010000:1.0:1789762766.357504:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762766.357512:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174c4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.357530:0:9038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174c4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.357541:0:9038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174c4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.357552:0:9038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174c4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762766.357602:0:9038:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762766.357622:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762766.357640:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174c4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.357663:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762766.357672:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174c4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.357693:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174c4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.357726:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174c4b lrc: 1/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.357745:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174c4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.357791:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762766.357795:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762766.357799:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762766.358714:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074634880:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1322us (1501us total) trans 4294967555 rc 0/0 00000100:00100000:1.0:1789762766.358736:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 101 00000004:00080000:3.0:1789762766.358756:0:3426:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37f6088000/2, rc 0, transno 4294967555 00000100:00100000:3.0:1789762766.358770:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074634880:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1789762766.412544:0:14356:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 239659622546ns, next at 239594619297ns, rc = 0 00000004:00080000:2.0:1789762766.412670:0:14356:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 239659751493ns, next at 200647167697ns, rc = 0 00000004:00080000:2.0:1789762766.412859:0:14356:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 239659940161ns, next at 0ns, rc = 0 00000004:00080000:2.0:1789762766.413091:0:14356:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 239660171484ns, next at 200647167449ns, rc = 0 00000004:00080000:1.0:1789762766.414008:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 8 committed 00000004:00080000:3.0:1789762766.414431:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 6 committed 00000004:00080000:0.0:1789762766.414884:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 8 committed 00000004:00080000:3.0:1789762766.415712:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 3 committed 00010000:00010000:2.0:1789762766.465027:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762767.360238:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789762767.360268:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fdfe0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074635008:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762767.360343:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074635264:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762767.360409:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074635648:0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.360426:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762767.360495:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074635136:0@lo:400:kworker.0 00000100:00100000:2.0:1789762767.360528:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762767.360537:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074635392:0@lo:400:kworker.0 00000100:00100000:1.0:1789762767.360603:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762767.360615:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762767.360625:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074636032:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762767.360698:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762767.360718:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074635904:0@lo:400:kworker.0 00000100:00100000:1.0:1789762767.360723:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074635520:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762767.360770:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074636544:0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.360786:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762767.360811:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762767.360839:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074636416:0@lo:400:kworker.0 00000100:00100000:1.0:1789762767.360884:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074636160:0@lo:400:kworker.0 00000100:00100000:2.0:1789762767.360894:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762767.360912:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074635008 00000100:00100000:3.0:1789762767.360954:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074635008 00000100:00100000:1.0:1789762767.361038:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762767.361045:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074636032 00000100:00100000:3.0:1789762767.361051:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074636032 00000100:00100000:3.0:1789762767.361112:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074635520 00000100:00100000:0.0:1789762767.361113:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074635392 00000100:00100000:3.0:1789762767.361119:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074635520 00000100:00100000:0.0:1789762767.361127:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074635392 00000100:00100000:2.0:1789762767.361131:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074635648 00000100:00100000:1.0:1789762767.361134:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074636672:0@lo:400:kworker.0 00000100:00100000:2.0:1789762767.361142:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074635648 00000100:00100000:3.0:1789762767.361204:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 122 00000100:00100000:1.0:1789762767.361217:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762767.361225:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074636544 00000100:00100000:2.0:1789762767.361231:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074635904 00000100:00100000:3.0:1789762767.361233:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074635008:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762767.361234:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074636544 00000100:00100000:2.0:1789762767.361242:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074635904 00000100:00100000:1.0:1789762767.361285:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00002000:00000024:3.0:1789762767.361286:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762767.361309:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:0.0:1789762767.361315:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074636160 00000100:00100000:1.0:1789762767.361316:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074636288:0@lo:400:kworker.0 00000100:00100000:0.0:1789762767.361325:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074636160 00000100:00100000:2.0:1789762767.361332:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074635648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.361367:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074635008:12345-0@lo:13:osp-pre-1-0.0 Request processed in 143us (947us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762767.361383:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 122 00000100:00100000:0.0:1789762767.361389:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:1.0:1789762767.361407:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762767.361410:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074635392:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.361426:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 123 00000100:00100000:3.0:1789762767.361443:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074636032:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762767.361462:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074636800:0@lo:400:kworker.0 00002000:00000024:3.0:1789762767.361464:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762767.361465:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074635648:12345-0@lo:400:kworker.0 Request processed in 139us (940us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762767.361491:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00100000:0.0:1789762767.361511:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074635392:12345-0@lo:400:kworker.0 Request processed in 105us (896us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762767.361526:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:2.0:1789762767.361532:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:3.0:1789762767.361561:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074636032:12345-0@lo:13:osp-pre-1-1.0 Request processed in 113us (857us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762767.361562:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074636288 00000100:00100000:2.0:1789762767.361562:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074635904:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762767.361563:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762767.361573:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074636288 00000100:00100000:3.0:1789762767.361589:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 123 00000100:00100000:3.0:1789762767.361625:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 124 00000100:00100000:0.0:1789762767.361645:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 293 00000100:00100000:3.0:1789762767.361647:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3424:x1876702074635520:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762767.361661:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074635136 00000100:00100000:2.0:1789762767.361662:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074635904:12345-0@lo:400:kworker.0 Request processed in 111us (876us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762767.361674:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074636544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762767.361675:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074635136 00002000:00000024:3.0:1789762767.361683:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762767.361686:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:2.0:1789762767.361777:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00100000:1.0:1789762767.361806:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 320 00000100:00100000:2.0:1789762767.361820:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074636160:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.361824:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3424:x1876702074635520:12345-0@lo:13:osp-pre-0-0.0 Request processed in 184us (1017us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762767.361840:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15c7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074635136:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762767.361841:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074636544:12345-0@lo:400:kworker.0 Request processed in 165us (946us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762767.361852:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 124 00000100:00100000:0.0:1789762767.361877:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 293 00000100:00100000:2.0:1789762767.361915:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074636160:12345-0@lo:400:kworker.0 Request processed in 103us (884us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762767.361918:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762767.361928:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 295 00000100:00080000:3.0:1789762767.361931:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762767.361942:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00000100:00080000:3.0:1789762767.361943:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762767.361991:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074636288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.362005:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762767.362026:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15c7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074635136:12345-0@lo:400:kworker.0 Request processed in 198us (1422us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762767.362031:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074636416 00000100:00100000:3.0:1789762767.362043:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074636928:0@lo:400:kworker.0 00000100:00100000:2.0:1789762767.362047:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074636416 00000100:00100000:1.0:1789762767.362055:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 320 00000100:00100000:0.0:1789762767.362071:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074636288:12345-0@lo:400:kworker.0 Request processed in 129us (669us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762767.362127:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762767.362146:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 295 00000100:00100000:2.0:1789762767.362163:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:2.0:1789762767.362186:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074636416:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.362203:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fdfe0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074635008:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762767.362204:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074636672 00000100:00100000:1.0:1789762767.362220:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074636672 00000100:00100000:3.0:1789762767.362259:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074635392:0@lo:400:kworker.0 00000100:00100000:2.0:1789762767.362281:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074636416:12345-0@lo:400:kworker.0 Request processed in 98us (274us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762767.362301:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:0.0:1789762767.362302:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074636800 00000100:00100000:3.0:1789762767.362304:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074635904:0@lo:400:kworker.0 00000100:00100000:1.0:1789762767.362306:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074636928 00000100:00100000:0.0:1789762767.362316:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074636800 00000100:00100000:1.0:1789762767.362319:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074636928 00000100:00100000:3.0:1789762767.362341:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074636416:0@lo:400:kworker.0 00000100:00100000:2.0:1789762767.362398:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 117 00000100:00100000:1.0:1789762767.362414:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 118 00000100:00080000:3.0:1789762767.362418:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762767.362428:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074636672:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789762767.362430:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762767.362439:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 119 00000100:00100000:1.0:1789762767.362446:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074636800:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.362460:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074636544:0@lo:400:kworker.0 00000100:00100000:0.0:1789762767.362471:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074636928:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.362485:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074635648:0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.362521:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074636032:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762767.362529:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074636672:12345-0@lo:400:kworker.0 Request processed in 112us (1317us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762767.362543:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 117 00000100:00100000:1.0:1789762767.362573:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074636800:12345-0@lo:400:kworker.0 Request processed in 135us (1015us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762767.362591:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 118 00000100:00100000:3.0:1789762767.362598:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074636928:0@lo:400:kworker.0 00000100:00100000:0.0:1789762767.362611:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074636928:12345-0@lo:400:kworker.0 Request processed in 142us (479us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762767.362629:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 119 00000100:00100000:3.0:1789762767.362674:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074636288:0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.362693:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074636800:0@lo:400:kworker.0 00000100:00100000:1.0:1789762767.362700:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762767.362712:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074635136:0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.362726:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074635520:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762767.362738:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074636160:0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.362756:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074636672:0@lo:400:kworker.0 00000100:00100000:1.0:1789762767.362782:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074635776:0@lo:400:kworker.0 00000100:00100000:2.0:1789762767.362807:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074635264 00000100:00100000:2.0:1789762767.362819:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074635264 00000100:00100000:1.0:1789762767.362840:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762767.362883:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 125 00000100:00100000:2.0:1789762767.362897:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074635264:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789762767.362916:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762767.362962:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074635776 00000100:00100000:0.0:1789762767.362975:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074635776 00000100:00100000:2.0:1789762767.363033:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074635264:12345-0@lo:13:osp-pre-0-1.0 Request processed in 138us (341us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762767.363043:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 125 00000100:00100000:0.0:1789762767.363051:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:0.0:1789762767.363071:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074635776:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762767.363148:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074635264:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762767.363169:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074635776:12345-0@lo:400:kworker.0 Request processed in 100us (326us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762767.363184:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074635776:0@lo:400:kworker.0 00000100:00100000:0.0:1789762767.363186:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00010000:00010000:2.0:1789762767.488115:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762768.512185:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762769.408286:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074637056:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762769.408412:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762769.408504:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074637056 00000100:00100000:3.0:1789762769.408519:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074637056 00000100:00100000:3.0:1789762769.408600:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:3.0:1789762769.408616:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074637056:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789762769.408668:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:3.0:1789762769.408749:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074637056:12345-0@lo:41:osp-pre-1-0.0 Request processed in 136us (342us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762769.408783:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:2.0:1789762769.408855:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074637056:0@lo:41:osp-pre-1-0.0 00010000:00010000:0.0:1789762769.536108:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762770.432081:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789762770.432105:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ca4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074637184:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762770.432174:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762770.432217:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074637184 00000100:00100000:0.0:1789762770.432226:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074637184 00000100:00100000:0.0:1789762770.432273:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 193 00000100:00100000:0.0:1789762770.432282:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074637184:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762770.432314:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762770.432360:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074637184:12345-0@lo:41:osp-pre-0-1.0 Request processed in 81us (187us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762770.432366:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 193 00000100:00100000:3.0:1789762770.432408:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ca4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074637184:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1789762770.560028:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762770.968596:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967555 is committed 00000100:00100000:0.0:1789762771.400028:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762771.400093:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762771.400102:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067305600 00000100:00100000:2.0:1789762771.400113:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067305600 00000100:00100000:0.0:1789762771.400136:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762771.400154:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067305472 00000100:00100000:3.0:1789762771.400160:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067305472 00000100:00080000:1.0:1789762771.400172:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762771.400179:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762771.400181:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067305728 00000100:00100000:2.0:1789762771.400185:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067305728 00000100:00100000:3.0:1789762771.400207:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 321 00000100:00100000:2.0:1789762771.400225:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 298 00000100:00100000:3.0:1789762771.400226:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15c7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067305472:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762771.400231:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762771.400237:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 299 00000100:00100000:2.0:1789762771.400243:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067305600:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762771.400254:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067305728:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762771.400298:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067305600:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 56us (274us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762771.400298:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15c7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067305472:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 74us (198us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762771.400300:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067305728:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 49us (162us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762771.400304:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762771.400319:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 321 00000100:00100000:2.0:1789762771.400321:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 298 00000100:00100000:1.0:1789762771.400329:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 299 00000100:00100000:2.0:1789762771.400344:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067305984 00000100:00100000:2.0:1789762771.400348:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067305984 00000100:00100000:1.0:1789762771.400352:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067306112 00000100:00100000:1.0:1789762771.400364:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067306112 00000100:00100000:2.0:1789762771.400387:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 300 00000100:00100000:2.0:1789762771.400400:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067305984:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762771.400401:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 301 00000100:00100000:3.0:1789762771.400415:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067306112:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:1.0:1789762771.400421:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762771.400426:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762771.400434:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067305984:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 37us (209us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762771.400449:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 300 00000100:00100000:1.0:1789762771.400469:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067305856 00000100:00100000:1.0:1789762771.400473:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067305856 00000100:00080000:0.0:1789762771.400506:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762771.400507:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 120 00000100:00100000:1.0:1789762771.400517:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067305856:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762771.400543:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067306112:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 130us (245us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762771.400550:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 301 00000100:00100000:1.0:1789762771.400564:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067305856:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 46us (387us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762771.400575:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 120 00010000:00010000:2.0:1789762771.583988:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762772.480150:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9fff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074637312:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762772.480191:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ca7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074637568:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762772.480246:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762772.480267:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074637696:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762772.480304:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762772.480345:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762772.480368:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789762772.480385:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074637440:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762772.480403:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074637312 00000100:00100000:1.0:1789762772.480413:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074637312 00000100:00100000:3.0:1789762772.480431:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074637568 00000100:00100000:3.0:1789762772.480441:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074637568 00000100:00100000:2.0:1789762772.480442:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762772.480512:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074637696 00000100:00100000:1.0:1789762772.480515:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074637440 00000100:00100000:3.0:1789762772.480519:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074637696 00000100:00100000:1.0:1789762772.480524:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074637440 00000100:00100000:2.0:1789762772.480532:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 126 00000100:00100000:2.0:1789762772.480562:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074637568:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762772.480570:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00000100:00100000:1.0:1789762772.480587:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 128 00000100:00100000:3.0:1789762772.480596:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074637312:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789762772.480612:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762772.480614:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074637696:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789762772.480644:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00002000:00000024:3.0:1789762772.480658:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762772.480681:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074638208:0@lo:400:kworker.0 00000100:00100000:2.0:1789762772.480704:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074637568:12345-0@lo:13:osp-pre-1-1.0 Request processed in 155us (402us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762772.480718:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 126 00000100:00100000:2.0:1789762772.480747:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 129 00000100:00100000:2.0:1789762772.480772:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074637440:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762772.480772:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074637312:12345-0@lo:13:osp-pre-0-1.0 Request processed in 178us (521us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762772.480780:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762772.480782:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074637696:12345-0@lo:13:osp-pre-1-0.0 Request processed in 170us (435us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762772.480802:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00002000:00000024:2.0:1789762772.480804:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762772.480815:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 128 00000100:00100000:0.0:1789762772.480828:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074638720:0@lo:400:kworker.0 00000100:00080000:1.0:1789762772.480879:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789762772.480889:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762772.480914:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074637440:12345-0@lo:13:osp-pre-0-0.0 Request processed in 152us (472us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762772.480916:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762772.480917:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074638080:0@lo:400:kworker.0 00000100:00100000:1.0:1789762772.480918:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074638208 00000100:00100000:1.0:1789762772.480929:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074638208 00000100:00100000:2.0:1789762772.480992:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 129 00000100:00100000:1.0:1789762772.481028:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074638720 00000100:00100000:1.0:1789762772.481038:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074638720 00000100:00100000:3.0:1789762772.481040:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762772.481122:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074637696:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762772.481136:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074637952:0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.481136:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074638592:0@lo:400:kworker.0 00000100:00100000:1.0:1789762772.481152:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:0.0:1789762772.481161:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:0.0:1789762772.481180:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074637824:0@lo:400:kworker.0 00000100:00100000:1.0:1789762772.481188:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074638208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.481220:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762772.481259:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9fff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074637312:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762772.481272:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762772.481289:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074638208:12345-0@lo:400:kworker.0 Request processed in 109us (515us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762772.481299:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967555 gen 2 00000100:00100000:1.0:1789762772.481301:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000004:00080000:3.0:1789762772.481309:0:3427:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37f6088000, transno 4294967555 00000100:00100000:2.0:1789762772.481310:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074638592 00000100:00100000:2.0:1789762772.481320:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074638592 00000100:00100000:1.0:1789762772.481347:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 303 00000100:00100000:3.0:1789762772.481350:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074637440:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762772.481370:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074638720:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762772.481377:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074638336:0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.481394:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ca7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074637568:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762772.481396:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:2.0:1789762772.481428:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074638592:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762772.481452:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074638720:12345-0@lo:400:kworker.0 Request processed in 85us (538us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762772.481454:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762772.481470:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 303 00000100:00100000:3.0:1789762772.481472:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074637824 00000100:00100000:3.0:1789762772.481480:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074637824 00000100:00100000:0.0:1789762772.481509:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074638848:0@lo:400:kworker.0 00000100:00100000:1.0:1789762772.481529:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074638080 00000100:00100000:2.0:1789762772.481539:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074638592:12345-0@lo:400:kworker.0 Request processed in 113us (316us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762772.481540:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074638080 00000100:00100000:3.0:1789762772.481541:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 322 00000100:00100000:0.0:1789762772.481560:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762772.481564:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15c5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074637824:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762772.481574:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00100000:1.0:1789762772.481602:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074638336 00000100:00100000:1.0:1789762772.481610:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074638336 00000100:00100000:2.0:1789762772.481615:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074638848 00000100:00100000:2.0:1789762772.481624:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074638848 00000100:00100000:0.0:1789762772.481631:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074638208:0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.481645:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15c5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074637824:12345-0@lo:400:kworker.0 Request processed in 83us (380us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762772.481658:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 322 00000100:00100000:1.0:1789762772.481665:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:0.0:1789762772.481667:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074638720:0@lo:400:kworker.0 00000100:00100000:1.0:1789762772.481688:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074638080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762772.481698:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 305 00000100:00100000:0.0:1789762772.481714:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074637824:0@lo:400:kworker.0 00000100:00100000:2.0:1789762772.481720:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074638848:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.481728:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762772.481782:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:3.0:1789762772.481807:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074638464:0@lo:400:kworker.0 00000100:00100000:1.0:1789762772.481813:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074638080:12345-0@lo:400:kworker.0 Request processed in 113us (760us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762772.481822:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074638336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762772.481834:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:2.0:1789762772.481846:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074638848:12345-0@lo:400:kworker.0 Request processed in 132us (290us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762772.481860:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 305 00000100:00100000:3.0:1789762772.481869:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762772.481884:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074637952 00000100:00100000:1.0:1789762772.481892:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074637952 00000100:00100000:2.0:1789762772.481903:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074638464 00000100:00100000:2.0:1789762772.481912:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074638464 00000100:00100000:0.0:1789762772.481927:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074638336:12345-0@lo:400:kworker.0 Request processed in 111us (477us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762772.482004:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 306 00000100:00100000:0.0:1789762772.482014:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00100000:3.0:1789762772.482020:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 307 00000100:00100000:2.0:1789762772.482024:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074637952:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.482045:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074638464:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789762772.482111:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789762772.482119:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762772.482150:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074637952:12345-0@lo:400:kworker.0 Request processed in 129us (427us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762772.482166:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 306 00000100:00100000:3.0:1789762772.482193:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074638464:12345-0@lo:400:kworker.0 Request processed in 152us (326us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762772.482206:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074637952:0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.482211:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 307 00000100:00100000:1.0:1789762772.482216:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074638336:0@lo:400:kworker.0 00000100:00100000:0.0:1789762772.482248:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074638464:0@lo:400:kworker.0 00000100:00100000:1.0:1789762772.482256:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074638848:0@lo:400:kworker.0 00000100:00100000:2.0:1789762772.482267:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074638080:0@lo:400:kworker.0 00000100:00100000:2.0:1789762772.482294:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074638592:0@lo:400:kworker.0 00010000:00010000:1.0:1789762772.608128:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762773.632107:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762774.528163:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789762774.528184:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074638976:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762774.528243:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762774.528279:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074638976 00000100:00100000:1.0:1789762774.528287:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074638976 00000100:00100000:1.0:1789762774.528347:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:1.0:1789762774.528356:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074638976:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762774.528390:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762774.528442:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074638976:12345-0@lo:41:osp-pre-1-0.0 Request processed in 88us (198us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762774.528449:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00100000:2.0:1789762774.528485:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074638976:0@lo:41:osp-pre-1-0.0 00010000:00010000:2.0:1789762774.656122:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762775.552151:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789762775.552181:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ca7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074639104:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762775.552252:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762775.552287:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074639104 00000100:00100000:0.0:1789762775.552297:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074639104 00000100:00100000:0.0:1789762775.552367:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:0.0:1789762775.552378:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9fff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074639104:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762775.552413:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762775.552459:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9fff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074639104:12345-0@lo:41:osp-pre-0-1.0 Request processed in 83us (212us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762775.552467:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:1.0:1789762775.552504:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ca7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074639104:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1789762775.680069:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762776.520496:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762776.520573:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762776.520596:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067306240 00000100:00100000:1.0:1789762776.520613:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067306240 00000100:00100000:0.0:1789762776.520631:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762776.520652:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067306368 00000100:00100000:2.0:1789762776.520661:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067306368 00000100:00100000:0.0:1789762776.520688:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762776.520704:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067306624 00000100:00100000:3.0:1789762776.520736:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067306624 00000100:00100000:1.0:1789762776.520739:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 323 00000100:00100000:2.0:1789762776.520750:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067306496 00000100:00100000:2.0:1789762776.520758:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067306496 00000100:00100000:1.0:1789762776.520760:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15c4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067306240:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762776.520762:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762776.520802:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067306880 00000100:00100000:2.0:1789762776.520824:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067306880 00000100:00100000:3.0:1789762776.520830:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 121 00000100:00100000:0.0:1789762776.520837:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762776.520851:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067306624:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762776.520860:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067306752 00000100:00100000:2.0:1789762776.520865:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067306752 00000100:00100000:0.0:1789762776.520899:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 308 00000100:00100000:1.0:1789762776.520902:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15c4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067306240:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 145us (415us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762776.520914:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067306624:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 66us (280us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762776.520921:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 323 00000100:00100000:3.0:1789762776.520933:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 121 00000100:00100000:0.0:1789762776.520947:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067306368:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762776.520969:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 309 00000100:00100000:2.0:1789762776.520991:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067306496:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762776.521002:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067306368:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 89us (427us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762776.521012:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 310 00000100:00100000:0.0:1789762776.521038:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 308 00000100:00100000:3.0:1789762776.521046:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067306880:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:1.0:1789762776.521064:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762776.521073:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 311 00000100:00100000:2.0:1789762776.521073:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067306496:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 87us (382us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762776.521075:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789762776.521087:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762776.521094:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 309 00000100:00100000:0.0:1789762776.521096:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067306752:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762776.521104:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067306880:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 80us (348us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762776.521114:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 310 00000100:00100000:0.0:1789762776.521162:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067306752:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 73us (328us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762776.521171:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 311 00010000:00010000:1.0:1789762776.704194:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762777.600093:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1789762777.600120:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ca4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074639360:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762777.600199:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fdfe0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074639872:0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.600218:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762777.600322:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762777.600337:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074639232:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762777.600393:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074640384:0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.600402:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762777.600448:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762777.600462:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074639744:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762777.600468:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074639488:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762777.600483:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074640896:0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.600540:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762777.600561:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074639360 00000100:00100000:3.0:1789762777.600570:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074639360 00000100:00100000:1.0:1789762777.600577:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762777.600610:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074640000:0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.600668:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762777.600676:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074639872 00000100:00100000:3.0:1789762777.600681:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074639232 00000100:00100000:0.0:1789762777.600686:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074640896 00000100:00100000:1.0:1789762777.600686:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074639872 00000100:00100000:3.0:1789762777.600688:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074639232 00000100:00100000:2.0:1789762777.600711:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074640512:0@lo:400:kworker.0 00000100:00100000:0.0:1789762777.600712:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074640896 00000100:00100000:3.0:1789762777.600725:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074639488 00000100:00100000:3.0:1789762777.600730:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074639488 00000100:00100000:2.0:1789762777.600761:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762777.600788:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 130 00000100:00100000:1.0:1789762777.600792:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074640384 00000100:00100000:1.0:1789762777.600798:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074640384 00000100:00100000:2.0:1789762777.600801:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074641024:0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.600802:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074639360:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762777.600827:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 122 00002000:00000024:3.0:1789762777.600855:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762777.600861:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 312 00000100:00100000:0.0:1789762777.600872:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074640896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.600872:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762777.600880:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074639872:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.600943:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074639360:12345-0@lo:13:osp-pre-1-1.0 Request processed in 117us (714us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762777.600960:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 130 00000100:00100000:2.0:1789762777.600994:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ca4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074639360:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762777.601001:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00100000:0.0:1789762777.601007:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074640896:12345-0@lo:400:kworker.0 Request processed in 165us (435us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762777.601008:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074639872:12345-0@lo:400:kworker.0 Request processed in 131us (693us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762777.601035:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074639232:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762777.601043:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 122 00000100:00100000:1.0:1789762777.601048:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 312 00000100:00100000:2.0:1789762777.601077:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 132 00000100:00100000:2.0:1789762777.601097:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074639488:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789762777.601097:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762777.601099:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074641024 00000100:00100000:0.0:1789762777.601112:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074641024 00000100:00100000:1.0:1789762777.601112:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074640512 00002000:00000024:2.0:1789762777.601116:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762777.601122:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074640512 00000100:00100000:0.0:1789762777.601163:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 123 00000100:00100000:3.0:1789762777.601182:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074639232:12345-0@lo:13:osp-pre-0-0.0 Request processed in 164us (783us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762777.601185:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074641024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.601187:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074639488:12345-0@lo:13:osp-pre-1-0.0 Request processed in 91us (644us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762777.601196:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 313 00000100:00100000:3.0:1789762777.601216:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00100000:2.0:1789762777.601222:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 132 00000100:00100000:1.0:1789762777.601232:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074640384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.601275:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 314 00000100:00100000:0.0:1789762777.601284:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074641024:12345-0@lo:400:kworker.0 Request processed in 111us (424us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762777.601295:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074640512:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789762777.601297:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762777.601303:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 123 00000100:00080000:3.0:1789762777.601309:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762777.601321:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074640384:12345-0@lo:400:kworker.0 Request processed in 94us (873us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762777.601323:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789762777.601338:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762777.601340:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074640000 00000100:00100000:1.0:1789762777.601345:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 313 00000100:00100000:0.0:1789762777.601347:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074640000 00000100:00100000:2.0:1789762777.601370:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074640512:12345-0@lo:400:kworker.0 Request processed in 81us (608us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762777.601373:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762777.601379:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 314 00000100:00100000:1.0:1789762777.601400:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074639744 00000100:00100000:1.0:1789762777.601406:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074639744 00000100:00100000:0.0:1789762777.601414:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:0.0:1789762777.601436:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074640000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.601448:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074640256:0@lo:400:kworker.0 00000100:00100000:1.0:1789762777.601456:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 133 00000100:00100000:2.0:1789762777.601462:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074639488:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762777.601472:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074639744:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789762777.601489:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762777.601491:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074640512:0@lo:400:kworker.0 00000100:00100000:0.0:1789762777.601503:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074640000:12345-0@lo:400:kworker.0 Request processed in 77us (843us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762777.601538:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:2.0:1789762777.601539:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074641024:0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.601539:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762777.601574:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074640256 00000100:00100000:1.0:1789762777.601578:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074639744:12345-0@lo:13:osp-pre-0-1.0 Request processed in 110us (209us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762777.601581:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074640256 00000100:00100000:2.0:1789762777.601584:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074639232:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762777.601589:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074640768:0@lo:400:kworker.0 00000100:00100000:1.0:1789762777.601597:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 133 00000100:00100000:0.0:1789762777.601656:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:2.0:1789762777.601666:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074640000:0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.601670:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762777.601692:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074640256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762777.601737:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074640384:0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.601744:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074639616:0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.601748:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074639744:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762777.601767:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074640256:12345-0@lo:400:kworker.0 Request processed in 96us (253us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762777.601768:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074640896:0@lo:400:kworker.0 00000100:00100000:0.0:1789762777.601780:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00100000:3.0:1789762777.601781:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074640256:0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.601808:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762777.601834:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fdfe0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074639872:0@lo:400:kworker.0 00000100:00100000:0.0:1789762777.601861:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074640768 00000100:00100000:3.0:1789762777.601861:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074639616 00000100:00100000:0.0:1789762777.601878:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074640768 00000100:00100000:3.0:1789762777.601879:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074639616 00000100:00100000:2.0:1789762777.601881:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074640128:0@lo:400:kworker.0 00000100:00100000:0.0:1789762777.601942:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 315 00000100:00100000:2.0:1789762777.601970:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762777.601976:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074640768:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.601983:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 324 00000100:00100000:1.0:1789762777.601985:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074640128 00000100:00100000:1.0:1789762777.601991:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074640128 00000100:00100000:3.0:1789762777.601998:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15c4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074639616:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.602005:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074640640:0@lo:400:kworker.0 00000100:00100000:1.0:1789762777.602073:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:2.0:1789762777.602079:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762777.602095:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15c4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074639616:12345-0@lo:400:kworker.0 Request processed in 91us (277us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762777.602103:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074640128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.602112:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 324 00000100:00100000:0.0:1789762777.602114:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074640768:12345-0@lo:400:kworker.0 Request processed in 156us (449us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762777.602125:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 315 00000100:00100000:2.0:1789762777.602130:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074641152:0@lo:400:kworker.0 00000100:00100000:0.0:1789762777.602154:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074640640 00000100:00100000:0.0:1789762777.602161:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074640640 00000100:00100000:1.0:1789762777.602187:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074640128:12345-0@lo:400:kworker.0 Request processed in 101us (230us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762777.602199:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762777.602204:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00100000:0.0:1789762777.602216:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 317 00000100:00100000:3.0:1789762777.602223:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074640768:0@lo:400:kworker.0 00000100:00100000:0.0:1789762777.602234:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074640640:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789762777.602250:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762777.602273:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074639616:0@lo:400:kworker.0 00000100:00080000:1.0:1789762777.602275:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762777.602281:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074641152 00000100:00100000:3.0:1789762777.602286:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074641152 00000100:00100000:0.0:1789762777.602315:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074640640:12345-0@lo:400:kworker.0 Request processed in 84us (245us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762777.602315:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074640128:0@lo:400:kworker.0 00000100:00100000:0.0:1789762777.602339:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 317 00000100:00100000:3.0:1789762777.602349:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 124 00000100:00100000:2.0:1789762777.602369:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074640640:0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.602369:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074641152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.602422:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074641152:12345-0@lo:400:kworker.0 Request processed in 63us (238us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762777.602428:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074641152:0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.602429:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 124 00010000:00010000:0.0:1789762777.728052:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762778.752030:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762779.648104:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074641280:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762779.648175:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762779.648228:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074641280 00000100:00100000:1.0:1789762779.648239:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074641280 00000100:00100000:1.0:1789762779.648297:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1789762779.648312:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074641280:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762779.648349:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762779.648398:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074641280:12345-0@lo:41:osp-pre-1-0.0 Request processed in 92us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762779.648404:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00100000:3.0:1789762779.648436:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074641280:0@lo:41:osp-pre-1-0.0 00010000:00010000:2.0:1789762779.776037:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762780.672126:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ca5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074641408:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762780.672188:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762780.672232:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074641408 00000100:00100000:0.0:1789762780.672241:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074641408 00000100:00100000:0.0:1789762780.672286:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 201 00000100:00100000:0.0:1789762780.672295:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074641408:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762780.672329:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762780.672374:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074641408:12345-0@lo:41:osp-pre-0-1.0 Request processed in 81us (185us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762780.672396:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 201 00000100:00100000:1.0:1789762780.672410:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ca5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074641408:0@lo:41:osp-pre-0-1.0 00010000:00010000:1.0:1789762780.800107:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762781.639907:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762781.639973:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067307136 00000100:00100000:2.0:1789762781.639979:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762781.639994:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067307136 00000100:00100000:2.0:1789762781.640025:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762781.640036:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067307008 00000100:00100000:3.0:1789762781.640042:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067307008 00000100:00100000:2.0:1789762781.640055:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00080000:0.0:1789762781.640063:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762781.640069:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067307264 00000100:00100000:1.0:1789762781.640074:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067307264 00000100:00100000:2.0:1789762781.640089:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762781.640089:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 325 00000100:00100000:0.0:1789762781.640101:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067307392 00000100:00100000:3.0:1789762781.640105:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15c5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067307008:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762781.640107:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067307392 00000100:00100000:1.0:1789762781.640126:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067307520 00000100:00100000:1.0:1789762781.640131:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067307520 00000100:00100000:2.0:1789762781.640133:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762781.640150:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 125 00000100:00100000:1.0:1789762781.640153:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067307648 00000100:00100000:1.0:1789762781.640156:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067307648 00000100:00100000:0.0:1789762781.640161:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067307392:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762781.640171:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15c5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067307008:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 70us (191us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762781.640178:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 318 00000100:00100000:3.0:1789762781.640179:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 325 00000100:00100000:1.0:1789762781.640192:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067307136:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762781.640203:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067307392:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 45us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762781.640211:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 125 00000100:00100000:1.0:1789762781.640226:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067307136:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 38us (329us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762781.640234:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 318 00000100:00080000:0.0:1789762781.640236:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762781.640240:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762781.640243:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762781.640255:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 319 00000100:00100000:1.0:1789762781.640265:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 320 00000100:00100000:3.0:1789762781.640266:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067307264:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762781.640282:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 321 00000100:00100000:1.0:1789762781.640282:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067307520:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762781.640292:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067307648:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762781.640332:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067307648:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 41us (200us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762781.640332:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067307520:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 55us (247us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762781.640341:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 320 00000100:00100000:0.0:1789762781.640347:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 321 00000100:00100000:3.0:1789762781.640406:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067307264:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 144us (352us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762781.640412:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 319 00010000:00010000:0.0:1789762781.824043:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762782.720096:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789762782.720116:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074641536:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762782.720125:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074641920:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762782.720140:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074641664:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762782.720187:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762782.720224:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074641536 00000100:00100000:3.0:1789762782.720231:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074641536 00000100:00100000:0.0:1789762782.720252:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762782.720270:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074642048:0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.720291:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074641664 00000100:00100000:0.0:1789762782.720294:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074641792:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762782.720296:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074641664 00000100:00100000:1.0:1789762782.720308:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762782.720340:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 134 00000100:00080000:2.0:1789762782.720350:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762782.720352:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074642560:0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.720354:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074641536:12345-0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1789762782.720356:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762782.720358:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:3.0:1789762782.720392:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762782.720399:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762782.720416:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074642304:0@lo:400:kworker.0 00000100:00100000:1.0:1789762782.720421:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762782.720428:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074642432:0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.720447:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074641536:12345-0@lo:13:osp-pre-0-1.0 Request processed in 98us (261us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762782.720457:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074643072:0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.720457:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 134 00000100:00100000:0.0:1789762782.720476:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762782.720476:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762782.720481:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074641792 00000100:00100000:3.0:1789762782.720486:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074641792 00000100:00100000:1.0:1789762782.720503:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074642560 00000100:00100000:1.0:1789762782.720508:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074642560 00000100:00100000:2.0:1789762782.720516:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074642944:0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.720523:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074641920 00000100:00100000:0.0:1789762782.720526:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cbad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074642816:0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.720530:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074641920 00000100:00100000:1.0:1789762782.720544:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074642304 00000100:00100000:2.0:1789762782.720545:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762782.720560:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074642304 00000100:00100000:3.0:1789762782.720569:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 135 00000100:00100000:0.0:1789762782.720577:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762782.720580:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074641664:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762782.720583:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074642432 00000100:00100000:2.0:1789762782.720595:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074642432 00002000:00000024:3.0:1789762782.720601:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762782.720606:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:1.0:1789762782.720616:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074642560:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762782.720622:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074643328:0@lo:400:kworker.0 00000100:00100000:2.0:1789762782.720634:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074642944 00000100:00100000:2.0:1789762782.720637:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074642944 00000100:00100000:3.0:1789762782.720643:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074641664:12345-0@lo:13:osp-pre-1-1.0 Request processed in 66us (396us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762782.720650:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1789762782.720664:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074642816 00000100:00100000:2.0:1789762782.720667:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074642816 00000100:00100000:3.0:1789762782.720670:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 136 00000100:00100000:3.0:1789762782.720679:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074641792:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762782.720687:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762782.720690:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074642560:12345-0@lo:400:kworker.0 Request processed in 75us (270us total) trans 0 rc 0/0 00002000:00000024:3.0:1789762782.720692:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762782.720698:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 322 00000100:00100000:1.0:1789762782.720702:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:2.0:1789762782.720710:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074642432:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762782.720727:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:1.0:1789762782.720736:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074642304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.720746:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074641792:12345-0@lo:13:osp-pre-1-0.0 Request processed in 67us (395us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762782.720750:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074641664:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762782.720755:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 136 00000100:00100000:2.0:1789762782.720774:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074642432:12345-0@lo:400:kworker.0 Request processed in 65us (300us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762782.720781:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 137 00000100:00100000:1.0:1789762782.720787:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074642304:12345-0@lo:400:kworker.0 Request processed in 50us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762782.720788:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074641792:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762782.720791:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 322 00000100:00100000:3.0:1789762782.720802:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7ada850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074641920:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762782.720809:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00002000:00000024:3.0:1789762782.720813:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762782.720818:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 323 00000100:00100000:2.0:1789762782.720829:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074642944:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762782.720842:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074642304:0@lo:400:kworker.0 00000100:00100000:1.0:1789762782.720847:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 324 00000100:00100000:3.0:1789762782.720857:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7ada850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074641920:12345-0@lo:13:osp-pre-0-0.0 Request processed in 57us (460us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762782.720863:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074642816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.720869:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 137 00000100:00100000:2.0:1789762782.720875:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074642944:12345-0@lo:400:kworker.0 Request processed in 47us (328us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762782.720882:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 323 00000100:00100000:0.0:1789762782.720887:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074643328 00000100:00100000:0.0:1789762782.720894:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074643328 00000100:00100000:1.0:1789762782.720914:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074642816:12345-0@lo:400:kworker.0 Request processed in 55us (338us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762782.720936:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 324 00000100:00100000:2.0:1789762782.720959:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074642432:0@lo:400:kworker.0 00000100:00100000:0.0:1789762782.720989:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 126 00000100:00100000:2.0:1789762782.720994:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074642944:0@lo:400:kworker.0 00000100:00100000:0.0:1789762782.721005:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074643328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762782.721019:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074641920:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762782.721034:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cbad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074642816:0@lo:400:kworker.0 00000100:00100000:1.0:1789762782.721057:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074643328:0@lo:400:kworker.0 00000100:00100000:0.0:1789762782.721064:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074643328:12345-0@lo:400:kworker.0 Request processed in 62us (377us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762782.721075:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 126 00000100:00100000:1.0:1789762782.721094:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762782.721114:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074643072 00000100:00100000:0.0:1789762782.721120:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074643072 00000100:00100000:2.0:1789762782.721129:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074642048 00000100:00100000:2.0:1789762782.721135:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074642048 00000100:00100000:1.0:1789762782.721140:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074641536:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762782.721172:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 325 00000100:00100000:1.0:1789762782.721172:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074642560:0@lo:400:kworker.0 00000100:00100000:2.0:1789762782.721179:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 326 00000100:00100000:1.0:1789762782.721192:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:0.0:1789762782.721193:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074643072:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762782.721193:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074642048:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762782.721204:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074642176:0@lo:400:kworker.0 00000100:00080000:3.0:1789762782.721223:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762782.721228:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789762782.721232:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789762782.721236:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762782.721249:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074642048:12345-0@lo:400:kworker.0 Request processed in 61us (940us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762782.721254:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762782.721261:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074643072:12345-0@lo:400:kworker.0 Request processed in 75us (168us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762782.721263:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 326 00000100:00100000:0.0:1789762782.721276:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 325 00000100:00100000:3.0:1789762782.721277:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074642176 00000100:00100000:3.0:1789762782.721283:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074642176 00000100:00100000:1.0:1789762782.721284:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074642688:0@lo:400:kworker.0 00000100:00100000:1.0:1789762782.721322:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762782.721322:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 326 00000100:00100000:0.0:1789762782.721333:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074642688 00000100:00100000:0.0:1789762782.721340:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074642688 00000100:00100000:2.0:1789762782.721341:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074642176:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762782.721346:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074643200:0@lo:400:kworker.0 00000100:00080000:3.0:1789762782.721369:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762782.721384:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762782.721392:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074642176:12345-0@lo:400:kworker.0 Request processed in 54us (140us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762782.721399:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 327 00000100:00100000:2.0:1789762782.721407:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 326 00000100:00100000:3.0:1789762782.721409:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074643200 00000100:00100000:0.0:1789762782.721414:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074642688:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762782.721415:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074643072:0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.721415:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074643200 00000100:00100000:1.0:1789762782.721436:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074642048:0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.721454:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00000100:00100000:1.0:1789762782.721465:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074642176:0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.721468:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074643200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762782.721478:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074642688:12345-0@lo:400:kworker.0 Request processed in 66us (158us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762782.721484:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074642688:0@lo:400:kworker.0 00000100:00100000:0.0:1789762782.721501:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 327 00000100:00100000:3.0:1789762782.721527:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074643200:12345-0@lo:400:kworker.0 Request processed in 63us (142us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762782.721530:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074643200:0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.721543:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00010000:00010000:2.0:1789762782.848017:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762783.872002:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762784.010624:0:14826:0:(nrs_tbf.c:670:nrs_tbf_cli_fini()) ost_io.0 NRS: TBF class fini: class@ffff9c37ef5cae00 key gid={0} rule@0xffff9c37c7d30400 gen 0 rate 10000 depth 3 token 2 deadline 196632661578 time 198278955689 state 0x6 ref 0 00000100:00100000:3.0:1789762784.010642:0:14826:0:(nrs_tbf.c:670:nrs_tbf_cli_fini()) ost_io.0 NRS: TBF class fini: class@ffff9c37dbbc3400 key gid={500} rule@0xffff9c37ff763c00 gen 0 rate 5 depth 3 token 0 deadline 238529279819 time 238529327636 state 0x6 ref 0 00000100:00100000:3.0:1789762784.018042:0:14826:0:(nrs_tbf.c:670:nrs_tbf_cli_fini()) ost_io.0 NRS: TBF class fini: class@ffff9c37ee4a6e00 key gid={0} rule@0xffff9c37c7d30800 gen 0 rate 10000 depth 3 token 2 deadline 196562749626 time 196562676302 state 0x6 ref 0 00000100:00100000:2.0:1789762784.768105:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762784.768128:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074643456:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762784.768191:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762784.768233:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074643456 00000100:00100000:1.0:1789762784.768242:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074643456 00000100:00100000:1.0:1789762784.768306:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:1.0:1789762784.768314:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074643456:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762784.768352:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762784.768409:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074643456:12345-0@lo:41:osp-pre-1-0.0 Request processed in 94us (216us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762784.768416:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:2.0:1789762784.768461:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074643456:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1789762784.896076:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762785.792115:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789762785.792139:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074643584:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789762785.792200:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762785.792242:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074643584 00000100:00100000:1.0:1789762785.792250:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074643584 00000100:00100000:1.0:1789762785.792296:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:1.0:1789762785.792306:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074643584:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789762785.792345:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789762785.792407:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074643584:12345-0@lo:41:osp-pre-0-1.0 Request processed in 101us (205us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762785.792443:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:0.0:1789762785.792455:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074643584:0@lo:41:osp-pre-0-1.0 00010000:00010000:0.0:1789762785.920070:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762786.760022:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762786.760054:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762786.760066:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067307776 00000100:00100000:3.0:1789762786.760076:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067307776 00000100:00100000:2.0:1789762786.760084:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762786.760115:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067308032 00000100:00100000:0.0:1789762786.760124:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067308032 00000100:00100000:3.0:1789762786.760128:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 327 00000100:00100000:3.0:1789762786.760140:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067307776:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762786.760151:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067307904 00000100:00100000:1.0:1789762786.760158:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067307904 00000100:00100000:0.0:1789762786.760185:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 328 00000100:00100000:1.0:1789762786.760198:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 329 00000100:00100000:0.0:1789762786.760199:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067308032:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762786.760212:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067307904:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762786.760232:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067307776:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 94us (216us total) trans 0 rc 0/0 00000100:00080000:2.0:1789762786.760236:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762786.760241:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 327 00000100:00080000:2.0:1789762786.760243:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762786.760247:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067308032:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 51us (187us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762786.760254:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067307904:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 44us (166us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762786.760261:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 328 00000100:00100000:1.0:1789762786.760268:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 329 00000100:00100000:0.0:1789762786.762296:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762786.762325:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762786.762346:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067308160 00000100:00100000:2.0:1789762786.762352:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067308160 00000100:00100000:1.0:1789762786.762371:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067308288 00000100:00100000:1.0:1789762786.762377:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067308288 00000100:00100000:0.0:1789762786.762384:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00080000:3.0:1789762786.762408:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762786.762413:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 102 00000100:00100000:0.0:1789762786.762420:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067308416 00000100:00100000:0.0:1789762786.762426:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067308416 00000100:00100000:2.0:1789762786.762427:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2394:x1876702067308160:12345-192.168.203.37@tcp:9:kworker.0 00000100:00100000:1.0:1789762786.762430:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 330 00000100:00100000:1.0:1789762786.762441:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067308288:12345-192.168.203.37@tcp:400:kworker.0 00000020:00000080:2.0:1789762786.762455:0:9037:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xdf5a7ac59f1742c7 00000100:00080000:3.0:1789762786.762475:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789762786.762483:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762786.762489:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 331 00000100:00080000:2.0:1789762786.762489:0:9037:0:(import.c:65:import_set_state_nolock()) ffff9c37d175d000 : changing import state from FULL to CLOSED 00000100:00100000:0.0:1789762786.762512:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067308416:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762786.762514:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067308288:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 71us (184us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762786.762548:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 330 00010000:00010000:3.0:1789762786.762550:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00000004:2.0:1789762786.762570:0:9037:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294967555 00000100:00100000:0.0:1789762786.762586:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067308416:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 81us (203us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762786.762596:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 331 00000100:00100000:2.0:1789762786.762876:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+2:2394:x1876702067308160:12345-192.168.203.37@tcp:9:kworker.0 Request processed in 454us (580us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762786.762884:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 102 00010000:00010000:2.0:1789762786.944102:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762787.840111:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074643840:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762787.840114:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074643968:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762787.840131:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074643712:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762787.840202:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762787.840245:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074644096:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762787.840311:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074643968 00000100:00100000:3.0:1789762787.840321:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762787.840327:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074643968 00000100:00100000:1.0:1789762787.840347:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074644480:0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.840388:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074644608:0@lo:400:kworker.0 00000100:00100000:1.0:1789762787.840416:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762787.840419:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074644096 00000100:00100000:0.0:1789762787.840426:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074644096 00000100:00100000:2.0:1789762787.840429:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762787.840443:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762787.840458:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074643712 00000100:00100000:0.0:1789762787.840464:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074643712 00000100:00100000:1.0:1789762787.840466:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074644992:0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.840500:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074645120:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.840504:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074644224:0@lo:400:kworker.0 00000100:00100000:0.0:1789762787.840511:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:1.0:1789762787.840518:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762787.840551:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074643968:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762787.840566:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762787.840569:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cbad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074645504:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.840588:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:0.0:1789762787.840593:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762787.840613:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762787.840631:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074644736:0@lo:400:kworker.0 00000100:00100000:0.0:1789762787.840665:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074643968:12345-0@lo:13:osp-pre-0-0.0 Request processed in 136us (464us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762787.840674:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074643968:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762787.840675:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00000100:00100000:2.0:1789762787.840690:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762787.840700:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 139 00000100:00100000:1.0:1789762787.840703:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:0.0:1789762787.840714:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074644096:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762787.840719:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cbb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074644352:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.840730:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074645248:0@lo:400:kworker.0 00000100:00080000:3.0:1789762787.840736:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00000024:0.0:1789762787.840737:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00080000:3.0:1789762787.840746:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762787.840758:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762787.840773:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 140 00000100:00100000:2.0:1789762787.840783:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762787.840789:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074644864:0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.840793:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074643712:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762787.840802:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074644096:12345-0@lo:13:osp-pre-1-1.0 Request processed in 91us (486us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762787.840811:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 139 00002000:00000024:3.0:1789762787.840817:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762787.840830:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762787.840859:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074645376:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.840862:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074644224 00000100:00100000:2.0:1789762787.840869:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074644224 00000100:00100000:3.0:1789762787.840882:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074643712:12345-0@lo:13:osp-pre-0-1.0 Request processed in 98us (460us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762787.840897:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 140 00000100:00100000:1.0:1789762787.840898:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762787.840922:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074644608 00000100:00100000:0.0:1789762787.840929:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074644608 00000100:00100000:2.0:1789762787.840975:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:1.0:1789762787.840985:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074645504 00000100:00100000:3.0:1789762787.840988:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074644992 00000100:00100000:2.0:1789762787.840992:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074644224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762787.840994:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074645504 00000100:00100000:3.0:1789762787.840996:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074644992 00000100:00100000:0.0:1789762787.841005:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074645120 00000100:00100000:0.0:1789762787.841011:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074645120 00000100:00100000:2.0:1789762787.841048:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074644224:12345-0@lo:400:kworker.0 Request processed in 61us (460us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762787.841048:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074644352 00000100:00100000:0.0:1789762787.841050:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074645248 00000100:00100000:3.0:1789762787.841055:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074644352 00000100:00100000:0.0:1789762787.841058:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074645248 00000100:00100000:2.0:1789762787.841065:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00100000:1.0:1789762787.841066:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074645376 00000100:00100000:1.0:1789762787.841073:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074645376 00000100:00100000:0.0:1789762787.841090:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074644864 00000100:00100000:0.0:1789762787.841096:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074644864 00000100:00100000:3.0:1789762787.841104:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:2.0:1789762787.841120:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 333 00000100:00100000:3.0:1789762787.841120:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074644608:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762787.841124:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 128 00000100:00100000:2.0:1789762787.841136:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074644992:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762787.841146:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:1.0:1789762787.841146:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074645504:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762787.841163:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074645120:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.841180:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074644608:12345-0@lo:400:kworker.0 Request processed in 65us (737us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762787.841188:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074644992:12345-0@lo:400:kworker.0 Request processed in 58us (679us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762787.841191:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00100000:2.0:1789762787.841197:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 333 00000100:00100000:1.0:1789762787.841214:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074645504:12345-0@lo:400:kworker.0 Request processed in 74us (601us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762787.841240:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 335 00000100:00100000:1.0:1789762787.841246:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 128 00000100:00100000:0.0:1789762787.841248:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074645120:12345-0@lo:400:kworker.0 Request processed in 65us (659us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762787.841257:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:3.0:1789762787.841263:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074645248:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762787.841272:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00100000:1.0:1789762787.841277:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 129 00000100:00100000:2.0:1789762787.841279:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074644352:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762787.841293:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074645376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762787.841343:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 337 00000100:00100000:3.0:1789762787.841344:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074645248:12345-0@lo:400:kworker.0 Request processed in 85us (555us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762787.841355:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074644352:12345-0@lo:400:kworker.0 Request processed in 85us (597us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762787.841359:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 335 00000100:00100000:0.0:1789762787.841364:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074644864:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762787.841367:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074645376:12345-0@lo:400:kworker.0 Request processed in 79us (473us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762787.841373:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00100000:1.0:1789762787.841383:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 129 00000100:00100000:1.0:1789762787.841419:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074644480 00000100:00100000:3.0:1789762787.841421:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074644608:0@lo:400:kworker.0 00000100:00100000:1.0:1789762787.841424:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074644480 00000100:00100000:0.0:1789762787.841427:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074644864:12345-0@lo:400:kworker.0 Request processed in 67us (595us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762787.841442:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 337 00000100:00100000:2.0:1789762787.841452:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074644224:0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.841452:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074645120:0@lo:400:kworker.0 00000100:00100000:1.0:1789762787.841469:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074644736 00000100:00100000:1.0:1789762787.841475:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074644736 00000100:00100000:0.0:1789762787.841482:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:2.0:1789762787.841488:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074645248:0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.841488:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074644096:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762787.841497:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074644480:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762787.841509:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:2.0:1789762787.841519:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074643712:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762787.841523:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d179ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074644736:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762787.841542:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074644480:12345-0@lo:400:kworker.0 Request processed in 48us (1129us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762787.841550:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00000100:00080000:2.0:1789762787.841580:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762787.841587:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762787.841592:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d179ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074644736:12345-0@lo:400:kworker.0 Request processed in 72us (906us total) trans 0 rc 0/0 00000100:00080000:2.0:1789762787.841595:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762787.841598:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762787.841609:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:3.0:1789762787.841618:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cbb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074644352:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.841645:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074643840 00000100:00100000:3.0:1789762787.841645:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074644864:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.841650:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074643840 00000100:00100000:1.0:1789762787.841663:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074644736:0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.841672:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074645376:0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.841688:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074644992:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.841692:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 141 00000100:00100000:3.0:1789762787.841700:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cbad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074645504:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.841701:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074643840:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762787.841714:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762787.841716:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074644480:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.841761:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074643840:12345-0@lo:13:osp-pre-1-0.0 Request processed in 59us (168us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762787.841772:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 141 00000100:00100000:0.0:1789762787.841790:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074643840:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789762787.968068:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762788.717036:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762788.717076:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067308544 00000100:00100000:3.0:1789762788.717087:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067308544 00000100:00080000:2.0:1789762788.717129:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762788.717144:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 161 00000100:00100000:3.0:1789762788.717161:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:11101:x1876702067308544:12345-192.168.203.37@tcp:101:bash.0 00010000:00010000:3.0:1789762788.717179:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762788.717194:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c52 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c53e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762788.717244:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c59 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762788.717256:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c59 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762788.717265:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c59 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762788.717293:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762788.717295:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762788.717308:0:6419:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c59 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762788.717326:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174c52 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c53e expref: 7 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762788.717337:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c59 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c53e expref: 6 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762788.717344:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762788.717346:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762788.717348:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7fc7e00, rc 0) 00000100:00100000:3.0:1789762788.717394:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:11101:x1876702067308544:12345-192.168.203.37@tcp:101:bash.0 Request processed in 236us (364us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762788.717409:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 161 00000100:00100000:1.0:1789762788.718285:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762788.718353:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067308672 00000100:00100000:2.0:1789762788.718359:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067308672 00000100:00100000:2.0:1789762788.718395:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 162 00000100:00100000:2.0:1789762788.718407:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:11101:x1876702067308672:12345-192.168.203.37@tcp:101:bash.0 00010000:00010000:2.0:1789762788.718417:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762788.718429:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11e00/0xdf5a7ac59f174c60 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c545 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718466:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174c67 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718477:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174c67 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718497:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174c67 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718509:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f174c6e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718514:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f174c6e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718518:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f174c6e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718661:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174c67 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718667:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174c67 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718671:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174c67 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718675:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174c67 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718679:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762788.718683:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174c67 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718688:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174c67 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718695:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762788.718701:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762788.718704:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174c67 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718708:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174c67 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718715:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f174c6e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718719:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f174c6e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718723:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f174c6e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718726:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762788.718728:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f174c6e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718731:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f174c6e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718735:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff9c37d3bee200) 00010000:00010000:2.0:1789762788.718737:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762788.718739:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f174c6e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718742:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11c00/0xdf5a7ac59f174c6e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718761:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11e00/0xdf5a7ac59f174c60 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c545 expref: 7 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718776:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762788.718778:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762788.718780:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11e00/0xdf5a7ac59f174c60 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c545 expref: 7 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762788.718785:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b11e00, rc 0) 00000100:00100000:2.0:1789762788.718814:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:11101:x1876702067308672:12345-192.168.203.37@tcp:101:bash.0 Request processed in 412us (530us total) trans 0 rc 301/301 00000100:00100000:2.0:1789762788.718818:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 162 00000001:00100000:3.0:1789762788.873113:0:8082:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000020:00000080:3.0:1789762788.873119:0:8082:0:(genops.c:885:class_export_put()) final put ffff9c37d175a800/fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000020:00000080:0.0:1789762788.873147:0:36:0:(genops.c:844:class_export_destroy()) destroying export ffff9c37d175a800/fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 for lustre-OST0000 00000020:00000080:0.0:1789762788.873197:0:36:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9c37d175d000 for lustre-OST0000 00010000:00010000:0.0:1789762788.992068:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1789762789.242207:0:15022:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 16:19:48 (1789762788) 00000100:00100000:1.0:1789762789.660970:0:15070:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:1.0:1789762789.660979:0:15070:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37f7b34500 rate 10000 gen 0 00000100:00100000:1.0:1789762789.661012:0:15070:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:1.0:1789762789.661015:0:15070:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37f7b34c00 rate 10000 gen 0 00000100:00100000:1.0:1789762789.661177:0:15070:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:1.0:1789762789.661185:0:15070:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:1.0:1789762789.661193:0:15070:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:1.0:1789762789.661200:0:15070:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:1.0:1789762789.661206:0:15070:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:1.0:1789762789.661211:0:15070:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37f7b34100 rate 20 gen 0 00000100:00100000:1.0:1789762789.661216:0:15070:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:1.0:1789762789.661221:0:15070:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:1.0:1789762789.661226:0:15070:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:1.0:1789762789.661230:0:15070:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37f7b34a00 rate 20 gen 0 00000100:00100000:1.0:1789762789.661369:0:15070:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:1.0:1789762789.661376:0:15070:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:1.0:1789762789.661382:0:15070:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:1.0:1789762789.661388:0:15070:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:1.0:1789762789.661394:0:15070:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:1.0:1789762789.661399:0:15070:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37f7b34b00 rate 10 gen 0 00000100:00100000:1.0:1789762789.661403:0:15070:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:1.0:1789762789.661408:0:15070:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:1.0:1789762789.661419:0:15070:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:1.0:1789762789.661424:0:15070:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37f7b34f00 rate 10 gen 0 00000100:00100000:0.0:1789762789.672178:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.672218:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067308800 00000100:00100000:0.0:1789762789.672223:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.672227:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067308800 00000100:00100000:2.0:1789762789.672262:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067309056 00000100:00100000:2.0:1789762789.672268:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067309056 00000100:00100000:1.0:1789762789.672297:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 163 00000100:00100000:1.0:1789762789.672311:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067308800:12345-192.168.203.37@tcp:41:mkdir.0 00000100:00100000:2.0:1789762789.672312:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 164 00000100:00100000:2.0:1789762789.672327:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13296:x1876702067309056:12345-192.168.203.37@tcp:36:mkdir.0 00000004:00000024:1.0:1789762789.672351:0:11692:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762789.672415:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d19f5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074645632:0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1789762789.672418:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067308800:12345-192.168.203.37@tcp:41:mkdir.0 Request processed in 110us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762789.672430:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 163 00000100:00080000:1.0:1789762789.672461:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762789.672475:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762789.672500:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074645632 00000100:00100000:0.0:1789762789.672505:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074645632 00000100:00100000:2.0:1789762789.672517:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f61c8540 going to sleep for 11 seconds 00000100:00100000:0.0:1789762789.672553:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:0.0:1789762789.672566:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074645632:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762789.672634:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074645632:12345-0@lo:1000:mdt00_005.0 Request processed in 71us (162us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762789.672641:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:2.0:1789762789.672655:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d19f5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074645632:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762789.672697:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d19f5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074645760:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762789.672720:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762789.672737:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f61c8540 going to sleep for 11 seconds 00000100:00100000:1.0:1789762789.672757:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074645760 00000100:00100000:1.0:1789762789.672763:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074645760 00000100:00100000:1.0:1789762789.672794:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:1.0:1789762789.672803:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074645760:12345-0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1789762789.673070:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074645760:12345-0@lo:1000:mdt00_005.0 Request processed in 266us (347us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789762789.673077:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d19f5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074645760:0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1789762789.673084:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00010000:00010000:2.0:1789762789.673126:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.673144:0:13389:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.673153:0:13389:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762789.673164:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d19f5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074645888:0@lo:101:mdt00_005.0 00000100:00100000:2.0:1789762789.673187:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762789.673212:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f61c8cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1789762789.673222:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074645888 00000100:00100000:0.0:1789762789.673234:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074645888 00000100:00100000:0.0:1789762789.673261:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:0.0:1789762789.673269:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074645888:12345-0@lo:101:mdt00_005.0 00010000:00010000:0.0:1789762789.673278:0:13302:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762789.673289:0:13302:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c7c lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f174c75 expref: -99 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.673309:0:13302:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c59 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c53e expref: 5 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.673318:0:13302:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c7c lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f174c75 expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.673336:0:13302:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c59 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c53e expref: 5 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.673346:0:13302:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c59 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c53e expref: 5 pid: 6419 timeout: 362 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762789.673354:0:13302:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:13302:1876702074646016:192.168.203.37@tcp:104: 00000100:00100000:0.0:1789762789.673399:0:13302:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da4540 going to sleep for 11 seconds 00000100:00100000:2.0:1789762789.674056:0:13302:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:13302:1876702074646016:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762789.674072:0:13302:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c7c lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174c75 expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.674080:0:13302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762789.674082:0:13302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762789.674085:0:13302:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c7c lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174c75 expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.674091:0:13302:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa6c00, rc 0) 00000100:00100000:2.0:1789762789.674120:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:13389:x1876702074645888:12345-0@lo:101:mdt00_005.0 Request processed in 853us (932us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762789.674125:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00100000:0.0:1789762789.674152:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d19f5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074645888:0@lo:101:mdt00_005.0 00010000:00010000:0.0:1789762789.674170:0:13389:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.674179:0:13389:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762789.674431:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.674465:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067309184 00000100:00100000:1.0:1789762789.674470:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067309184 00010000:00010000:1.0:1789762789.674486:0:6406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c59 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c53e expref: 6 pid: 6419 timeout: 362 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762789.674511:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 63 00000100:00100000:1.0:1789762789.674518:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067309184:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762789.674527:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762789.674532:0:6406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c59 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c53e expref: 6 pid: 6419 timeout: 362 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.674548:0:6406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c59 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c53e expref: 6 pid: 6419 timeout: 362 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.674563:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7e00/0xdf5a7ac59f174c59 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c53e expref: 6 pid: 6419 timeout: 362 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.674573:0:6406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762789.674578:0:6406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c7c lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174c75 expref: 6 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.674586:0:6406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c7c lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f174c75 expref: 6 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.674593:0:6406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c7c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f174c75 expref: 6 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.674613:0:6406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c7c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174c75 expref: 6 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762789.674621:0:6406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6406:1876702074646144:0@lo:105: 00000100:00100000:1.0:1789762789.674649:0:6406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762789.674677:0:6406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c9fc8b40 going to sleep for 11 seconds 00000100:00100000:2.0:1789762789.674692:0:6402:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1789762789.674702:0:6402:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074646144 00000100:00100000:2.0:1789762789.674706:0:6402:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074646144 00000100:00100000:2.0:1789762789.674735:0:6402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 41 00000100:00100000:2.0:1789762789.674742:0:6402:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fda0dc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6406:x1876702074646144:12345-0@lo:105: 00010000:00010000:2.0:1789762789.674758:0:6402:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.674765:0:6402:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.674772:0:6402:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.674777:0:6402:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.674782:0:6402:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.674789:0:6402:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.674804:0:6402:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c37e1b11400) 00010000:00010000:0.0:1789762789.674818:0:13389:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.674827:0:13389:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762789.674838:0:6402:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fda0dc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6406:x1876702074646144:12345-0@lo:105: Request processed in 95us (186us total) trans 0 rc 0/0 00000004:00080000:0.0:1789762789.674842:0:13389:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1789762789.674847:0:6402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 41 00000100:00100000:1.0:1789762789.674867:0:6406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6406:1876702074646144:0@lo:105: 00010000:00010000:0.0:1789762789.674870:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.674877:0:13389:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.674884:0:13389:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.674888:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762789.674897:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae2680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074646272:0@lo:101:mdt00_005.0 00000100:00100000:1.0:1789762789.674964:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067309184:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 445us (532us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762789.674965:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762789.674984:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 63 00000100:00100000:0.0:1789762789.674988:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da4e40 going to sleep for 11 seconds 00000100:00100000:2.0:1789762789.675024:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074646272 00000100:00100000:2.0:1789762789.675028:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074646272 00000100:00100000:2.0:1789762789.675054:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:2.0:1789762789.675061:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2731f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:13389:x1876702074646272:12345-0@lo:101:mdt00_005.0 00010000:00010000:2.0:1789762789.675068:0:6423:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762789.675096:0:6423:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174c8a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f174c83 expref: -99 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.675116:0:6423:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174c8a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f174c83 expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.675125:0:6423:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174c8a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174c83 expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.675130:0:6423:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762789.675132:0:6423:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762789.675134:0:6423:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b10e00, rc 0) 00000100:00100000:2.0:1789762789.675166:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2731f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074646272:12345-0@lo:101:mdt00_005.0 Request processed in 107us (205us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762789.675171:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00100000:0.0:1789762789.675193:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae2680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074646272:0@lo:101:mdt00_005.0 00010000:00010000:0.0:1789762789.675203:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f174c8a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.675210:0:13389:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174c8a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762789.675221:0:13389:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1789762789.675246:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2732300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074646400:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762789.675270:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762789.675288:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da4540 going to sleep for 11 seconds 00000100:00100000:2.0:1789762789.675300:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074646400 00000100:00100000:2.0:1789762789.675304:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074646400 00000100:00100000:2.0:1789762789.675335:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:2.0:1789762789.675343:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2732680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074646400:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762789.675392:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2732680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074646400:12345-0@lo:1000:mdt00_005.0 Request processed in 50us (120us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762789.675399:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:0.0:1789762789.675403:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2732300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074646400:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762789.675435:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2732300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074646528:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762789.675453:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762789.675470:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da4540 going to sleep for 11 seconds 00000100:00100000:1.0:1789762789.675484:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074646528 00000100:00100000:1.0:1789762789.675489:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074646528 00000100:00100000:1.0:1789762789.675514:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789762789.675521:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074646528:12345-0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1789762789.675662:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074646528:12345-0@lo:1000:mdt00_005.0 Request processed in 140us (206us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789762789.675669:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:0.0:1789762789.675678:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2732300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074646528:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762789.675734:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2732300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074646656:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762789.675754:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762789.675771:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da4540 going to sleep for 11 seconds 00000100:00100000:2.0:1789762789.675781:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074646656 00000100:00100000:2.0:1789762789.675785:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074646656 00000100:00100000:2.0:1789762789.675812:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1789762789.675820:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ebaa0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074646656:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762789.676032:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ebaa0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074646656:12345-0@lo:1000:mdt00_005.0 Request processed in 214us (278us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789762789.676040:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:0.0:1789762789.676054:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2732300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074646656:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762789.676101:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2732300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074646784:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762789.676129:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762789.676145:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da43c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762789.676148:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074646784 00000100:00100000:2.0:1789762789.676153:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074646784 00000100:00100000:2.0:1789762789.676177:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:2.0:1789762789.676184:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074646784:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762789.676234:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074646784:12345-0@lo:1000:mdt00_005.0 Request processed in 52us (104us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762789.676240:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00100000:0.0:1789762789.676251:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2732300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074646784:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762789.676303:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2732300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074646912:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762789.676324:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762789.676340:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da43c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762789.676342:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074646912 00000100:00100000:2.0:1789762789.676346:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074646912 00000100:00100000:2.0:1789762789.676368:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:2.0:1789762789.676375:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074646912:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762789.676427:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074646912:12345-0@lo:1000:mdt00_005.0 Request processed in 52us (100us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762789.676431:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2732300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074646912:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762789.676442:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00100000:0.0:1789762789.676472:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffc380 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074647040:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762789.676494:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762789.676510:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da43c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762789.676571:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074647040 00000100:00100000:3.0:1789762789.676579:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074647040 00000100:00100000:3.0:1789762789.676620:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:3.0:1789762789.676632:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074647040:12345-0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762789.676707:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074647040:12345-0@lo:1000:mdt00_005.0 Request processed in 79us (210us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762789.676715:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:0.0:1789762789.676734:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffc380 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074647040:0@lo:1000:mdt00_005.0 00000040:00080000:0.0:1789762789.678134:0:13389:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].5, 984 off36568 00000040:00080000:0.0:1789762789.678293:0:13389:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].5, 1136 off37184 00000004:00080000:0.0:1789762789.678298:0:13389:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c3801da4540 with version 12 00000004:00080000:2.0:1789762789.678330:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 12 rpc_version 12 00000020:00100000:2.0:1789762789.678352:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1789762789.678358:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c36d19f7800 x1876702074647296/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2416/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762789.678371:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d19f7800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074647296:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762789.678403:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762789.678427:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074647296 00000100:00100000:2.0:1789762789.678430:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f61c8f00 going to sleep for 11 seconds 00000100:00100000:0.0:1789762789.678432:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074647296 00000100:00100000:0.0:1789762789.678464:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:0.0:1789762789.678470:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7224:x1876702074647296:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762789.679188:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7224:x1876702074647296:12345-0@lo:1000:osp_up0-1.0 Request processed in 718us (785us total) trans 4294967365 rc 0/0 00000100:00100000:0.0:1789762789.679196:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:2.0:1789762789.679227:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d19f7800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074647296:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1789762789.679262:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c91 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.679272:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c91 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.679281:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c91 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762789.679291:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e86e3e30 reply state 000000006fc71a04 transno 4294967359 ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c91 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.679299:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c91 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.679304:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:0.0:1789762789.679306:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762789.679309:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762789.679312:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xb:0x0]: returning size 16384 00000004:00080000:0.0:1789762789.679316:0:13389:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1789762789.679330:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.679337:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.679342:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.679360:0:13389:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1789762789.679363:0:13389:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1789762789.679367:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f174c8a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.679373:0:13389:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f174c8a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.679375:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762789.679384:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.679394:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11400 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:2.0:1789762789.679409:0:6407:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762789.679421:0:6407:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762789.679428:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13296:x1876702067309056:12345-192.168.203.37@tcp:36:mkdir.0 Request processed in 7102us (7203us total) trans 4294967359 rc 0/0 00000100:00100000:0.0:1789762789.679441:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 164 00010000:00010000:2.0:1789762789.679459:0:6407:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.679468:0:6407:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762789.679497:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762789.679502:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074647424:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762789.679511:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37e1b11400/0xdf5a7ac59f174c75 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f174c7c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.679525:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762789.679528:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762789.679530:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762789.679535:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762789.679564:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074647424 00000100:00100000:2.0:1789762789.679570:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074647424 00000100:00100000:2.0:1789762789.679611:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21 00000100:00100000:2.0:1789762789.679623:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3424:x1876702074647424:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762789.679633:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762789.679646:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174c7c lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f174c75 expref: 9 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.679658:0:8070:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762789.679661:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762789.679702:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074647424:12345-0@lo:103:ldlm_bl.0 Request processed in 84us (168us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762789.679708:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21 00000100:00100000:0.0:1789762789.679719:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074647424:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762789.684740:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762789.684809:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067309312 00000100:00100000:2.0:1789762789.684815:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067309312 00000100:00100000:2.0:1789762789.684854:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 165 00000100:00100000:2.0:1789762789.684865:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2732300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:13297:x1876702067309312:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762789.684873:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762789.684883:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f174c98 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c54c expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.684961:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f174c9f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.684975:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f174c9f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.684985:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f174c9f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762789.685009:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762789.685013:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762789.685032:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f174c9f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.685052:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f174c98 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c54c expref: 7 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.685064:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f174c9f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c54c expref: 6 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.685080:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762789.685082:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762789.685084:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b10200, rc 0) 00000100:00100000:2.0:1789762789.685119:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2732300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13297:x1876702067309312:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 257us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762789.685124:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 165 00000100:00100000:1.0:1789762789.686103:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762789.686147:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067309568 00000100:00100000:0.0:1789762789.686153:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067309568 00000100:00100000:0.0:1789762789.686189:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 166 00000100:00100000:0.0:1789762789.686197:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13297:x1876702067309568:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762789.686212:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762789.686214:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1789762789.686226:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f174c9f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c54c expref: 6 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.686236:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762789.686238:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762789.686246:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6600/0xdf5a7ac59f174ca6 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c553 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.686304:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174cad lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.686311:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174cad lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.686318:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174cad lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.686325:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.686332:0:6419:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174c8a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174c83 expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.686337:0:6419:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.686356:0:6419:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174c8a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xdf5a7ac59f174c83 expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762789.686376:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff882680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6419:1876702074647552:0@lo:104: 00000100:00100000:0.0:1789762789.686400:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762789.686426:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c996a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762789.686437:0:6449:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074647552 00000100:00100000:1.0:1789762789.686442:0:6449:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074647552 00000100:00100000:1.0:1789762789.686473:0:6449:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 42 00000100:00100000:1.0:1789762789.686480:0:6449:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6419:x1876702074647552:12345-0@lo:104: 00010000:00010000:1.0:1789762789.686489:0:6449:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f174c8a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.686534:0:6449:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1789762789.686540:0:6449:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6419:x1876702074647552:12345-0@lo:104: Request processed in 60us (138us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762789.686545:0:6449:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 42 00010000:00010000:2.0:1789762789.686548:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:0.0:1789762789.686558:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff882680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6419:1876702074647552:0@lo:104: 00010000:00010000:2.0:1789762789.686559:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f174c8a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.686574:0:6419:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.686574:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7600 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:2.0:1789762789.686584:0:6408:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xdf5a7ac59f174c8a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762789.686596:0:6408:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xdf5a7ac59f174c8a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.686622:0:6408:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f174c8a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.686627:0:6408:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762789.686645:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f174c8a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.686651:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7600/0xdf5a7ac59f174c83 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f174c8a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762789.686654:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789762789.686666:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074647680:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762789.686678:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762789.686682:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762789.686684:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762789.686698:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762789.686725:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074647680 00000100:00100000:1.0:1789762789.686731:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074647680 00010000:00010000:1.0:1789762789.686749:0:6404:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174c8a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xdf5a7ac59f174c83 expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762789.686773:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 64 00000100:00100000:1.0:1789762789.686780:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074647680:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762789.686794:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1789762789.696302:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967365 is committed 00010000:00010000:1.0:1789762789.697960:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174c8a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xdf5a7ac59f174c83 expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.697976:0:6404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee500) 00010000:00010000:1.0:1789762789.697980:0:6404:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.697987:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.697992:0:6404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.698009:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762789.698030:0:6419:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698043:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762789.698056:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074647680:12345-0@lo:103:ldlm_bl.0 Request processed in 11275us (11355us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762789.698066:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 64 00000100:00100000:2.0:1789762789.698084:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074647680:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762789.698200:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6419:1876702074647808:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789762789.698235:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762789.698258:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c96a7180 going to sleep for 11 seconds 00000100:00100000:3.0:1789762789.698261:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074647808 00000100:00100000:3.0:1789762789.698267:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074647808 00000100:00100000:3.0:1789762789.698299:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:3.0:1789762789.698308:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6419:x1876702074647808:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762789.698370:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6419:x1876702074647808:12345-0@lo:1000:mdt00_001.0 Request processed in 64us (135us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762789.698376:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00000100:00100000:0.0:1789762789.698405:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5b480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6419:1876702074647808:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1789762789.698439:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174cbb lrc: 3/1,0 mode: --/PR res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698448:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174cbb lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698454:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174cbb lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698470:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174cad lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698476:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174cad lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698480:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174cad lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698485:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174cad lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698489:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762789.698492:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174cad lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698497:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174cad lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698505:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762789.698507:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762789.698509:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174cad lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698513:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174cad lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698521:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698526:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698536:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698540:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762789.698542:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698546:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698551:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee500) 00010000:00010000:0.0:1789762789.698553:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762789.698555:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698558:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174cb4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698583:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6600/0xdf5a7ac59f174ca6 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c553 expref: 7 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698593:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174cbb lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c553 expref: 6 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.698598:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125ac00) 00010000:00010000:0.0:1789762789.698600:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762789.698602:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa6800, rc 0) 00000100:00100000:0.0:1789762789.698637:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13297:x1876702067309568:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 12441us (12537us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762789.698651:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 166 00000100:00100000:0.0:1789762789.699802:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.699840:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067309824 00000100:00100000:1.0:1789762789.699845:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067309824 00000100:00100000:1.0:1789762789.699876:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 167 00000100:00100000:1.0:1789762789.699882:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13297:x1876702067309824:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:1.0:1789762789.699894:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762789.699906:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174cc2 lrc: 2/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c55a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762789.700010:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xb:0x0]: returning size 16384 00010000:00010000:1.0:1789762789.700029:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174cc2 lrc: 1/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c55a expref: 8 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.700053:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:1.0:1789762789.700056:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762789.700061:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f174cc2 lrc: 0/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c55a expref: 8 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.700073:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94800, rc 0) 00000100:00100000:1.0:1789762789.700107:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13297:x1876702067309824:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 225us (306us total) trans 0 rc 301/301 00000100:00100000:1.0:1789762789.700112:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 167 00000100:00100000:1.0:1789762789.700909:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762789.701021:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067309952 00000100:00100000:3.0:1789762789.701027:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067309952 00000100:00100000:3.0:1789762789.701067:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 168 00000100:00100000:3.0:1789762789.701077:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13297:x1876702067309952:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:3.0:1789762789.701088:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762789.701101:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7000/0xdf5a7ac59f174cc9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c561 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.701142:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174cd0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.701160:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174cd0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.701169:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174cd0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762789.701190:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762789.701194:0:7744:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762789.701209:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174cd0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.701229:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7000/0xdf5a7ac59f174cc9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c561 expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.701243:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174cd0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c561 expref: 7 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.701250:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762789.701252:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762789.701254:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7fc6600, rc 0) 00000100:00100000:3.0:1789762789.701284:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13297:x1876702067309952:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 210us (376us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762789.701290:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 168 00000100:00100000:0.0:1789762789.702321:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.702361:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067310208 00000100:00100000:1.0:1789762789.702367:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067310208 00000100:00100000:1.0:1789762789.702397:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 169 00000100:00100000:1.0:1789762789.702406:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13297:x1876702067310208:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:1.0:1789762789.702414:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762789.702422:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f174cd7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c568 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702457:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174cde lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702465:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174cde lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702472:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174cde lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702490:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f174ce5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702496:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f174ce5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702501:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f174ce5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702635:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94400/0xdf5a7ac59f174cec lrc: 3/1,0 mode: --/PR res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702646:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94400/0xdf5a7ac59f174cec lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702654:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94400/0xdf5a7ac59f174cec lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702665:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174cde lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702673:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174cde lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702681:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174cde lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702688:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762789.702693:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174cde lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702701:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174cde lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702719:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762789.702723:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762789.702727:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174cde lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702734:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174cde lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702748:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f174ce5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702756:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f174ce5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702763:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f174ce5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702769:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762789.702773:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f174ce5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702780:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f174ce5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702791:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743700) 00010000:00010000:1.0:1789762789.702794:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762789.702798:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f174ce5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702804:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f174ce5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702856:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f174cd7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c568 expref: 9 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702874:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94400/0xdf5a7ac59f174cec lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c568 expref: 8 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.702882:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125ac00) 00010000:00010000:1.0:1789762789.702885:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789762789.702888:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94400, rc 0) 00000100:00100000:1.0:1789762789.702978:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13297:x1876702067310208:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 570us (655us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762789.702988:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 169 00000100:00100000:0.0:1789762789.704396:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762789.704428:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067310464 00000100:00100000:0.0:1789762789.704434:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762789.704437:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067310464 00000100:00100000:0.0:1789762789.704463:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067310592 00000100:00100000:0.0:1789762789.704468:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067310592 00000100:00100000:2.0:1789762789.704494:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 170 00000100:00100000:0.0:1789762789.704510:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 22 00000100:00100000:2.0:1789762789.704511:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13297:x1876702067310464:12345-192.168.203.37@tcp:101:lfs.0 00000100:00100000:0.0:1789762789.704522:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067310592:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762789.704524:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762789.704532:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762789.704539:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174cf3 lrc: 2/0,0 mode: --/CR res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c56f expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.704546:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94400/0xdf5a7ac59f174cec lrc: 0/0,0 mode: --/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c568 expref: 8 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.704560:0:9878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125ac00) 00010000:00010000:0.0:1789762789.704564:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762789.704588:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067310592:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 70us (158us total) trans 0 rc 0/0 00000004:00200000:2.0:1789762789.704588:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xb:0x0]: returning size 16384 00000100:00100000:0.0:1789762789.704597:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 22 00010000:00010000:2.0:1789762789.704686:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174cf3 lrc: 1/0,0 mode: --/CR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c56f expref: 9 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.704696:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:2.0:1789762789.704698:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762789.704702:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174cf3 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c56f expref: 9 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.704713:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b11400, rc 0) 00000100:00100000:2.0:1789762789.704746:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13297:x1876702067310464:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 243us (352us total) trans 4294967360 rc 301/301 00000100:00100000:2.0:1789762789.704753:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 170 00000100:00100000:2.0:1789762789.705876:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762789.705908:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067310848 00000100:00100000:2.0:1789762789.705914:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067310848 00000100:00100000:2.0:1789762789.706000:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 171 00000100:00100000:2.0:1789762789.706015:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13297:x1876702067310848:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:2.0:1789762789.706063:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f174d01 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.706077:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f174d01 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.706089:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f174d01 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762789.706279:0:6418:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000715a9737 reply state 00000000d58ce267 transno 4294967361 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f174d01 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.706295:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f174d01 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.706303:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:2.0:1789762789.706307:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:2.0:1789762789.706310:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762789.706317:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xb:0x0]: returning size 16384 00000100:00100000:2.0:1789762789.706372:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:13297:x1876702067310848:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 361us (496us total) trans 4294967361 rc 0/0 00000100:00100000:2.0:1789762789.706381:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 171 00000100:00100000:0.0:1789762789.707165:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762789.707186:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067311104 00000100:00100000:2.0:1789762789.707191:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067311104 00000100:00100000:2.0:1789762789.707224:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 41 00000100:00100000:2.0:1789762789.707231:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:13297:x1876702067311104:12345-192.168.203.37@tcp:35:lfs.0 00000100:00100000:2.0:1789762789.707364:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:13297:x1876702067311104:12345-192.168.203.37@tcp:35:lfs.0 Request processed in 132us (198us total) trans 4294967362 rc 0/0 00000100:00100000:2.0:1789762789.707373:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 41 00000100:00100000:0.0:1789762789.711695:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762789.711733:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067311232 00000100:00100000:2.0:1789762789.711738:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067311232 00000100:00100000:2.0:1789762789.711769:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 172 00000100:00100000:2.0:1789762789.711776:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:13298:x1876702067311232:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:2.0:1789762789.711787:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762789.711801:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d08 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c576 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.711838:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11600/0xdf5a7ac59f174d0f lrc: 3/1,0 mode: --/PR res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.711847:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11600/0xdf5a7ac59f174d0f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.711855:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11600/0xdf5a7ac59f174d0f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762789.711873:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xb:0x0]: returning size 16384 00000004:00200000:2.0:1789762789.711875:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762789.711885:0:6419:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11600/0xdf5a7ac59f174d0f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.711902:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d08 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c576 expref: 13 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.712223:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11600/0xdf5a7ac59f174d0f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c576 expref: 12 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.712236:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:2.0:1789762789.712239:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762789.712241:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b11600, rc 0) 00000100:00100000:2.0:1789762789.712288:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:13298:x1876702067311232:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 512us (595us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762789.712296:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 172 00000100:00100000:0.0:1789762789.713978:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762789.714019:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067311488 00000100:00100000:0.0:1789762789.714027:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067311488 00000100:00100000:0.0:1789762789.714081:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 173 00000100:00100000:0.0:1789762789.714093:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:13298:x1876702067311488:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:0.0:1789762789.714137:0:6420:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762789.714163:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11600/0xdf5a7ac59f174d0f lrc: 0/0,0 mode: --/PR res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c576 expref: 12 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.714179:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:0.0:1789762789.714182:0:6420:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762789.714206:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7e00/0xdf5a7ac59f174d16 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.714217:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7e00/0xdf5a7ac59f174d16 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.714224:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7e00/0xdf5a7ac59f174d16 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762789.714427:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000582cc6a0 reply state 000000006d74b6a4 transno 4294967363 ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7e00/0xdf5a7ac59f174d16 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.714438:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7e00/0xdf5a7ac59f174d16 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.714443:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:0.0:1789762789.714445:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:0.0:1789762789.714448:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762789.714454:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xb:0x0]: returning size 16384 00000100:00100000:0.0:1789762789.714517:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:13298:x1876702067311488:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 427us (599us total) trans 4294967363 rc 0/0 00000100:00100000:0.0:1789762789.714524:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 173 00000100:00100000:0.0:1789762789.729642:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762789.729678:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067311616 00000100:00100000:3.0:1789762789.729686:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067311616 00000100:00100000:3.0:1789762789.729731:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 174 00000100:00100000:3.0:1789762789.729740:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13302:x1876702067311616:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762789.729754:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762789.729772:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7000/0xdf5a7ac59f174d1d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c57d expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.729816:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174d24 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.729825:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174d24 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.729832:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174d24 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.729842:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7600/0xdf5a7ac59f174d2b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.729847:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7600/0xdf5a7ac59f174d2b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.729851:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7600/0xdf5a7ac59f174d2b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730069:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6e00/0xdf5a7ac59f174d32 lrc: 3/1,0 mode: --/PR res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730076:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6e00/0xdf5a7ac59f174d32 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730081:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6e00/0xdf5a7ac59f174d32 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762789.730095:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6e00/0xdf5a7ac59f174d32 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730105:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174d24 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730114:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174d24 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730119:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174d24 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730123:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174d24 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730127:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7fc7200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762789.730130:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174d24 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730135:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174d24 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730142:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762789.730144:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762789.730148:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174d24 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730152:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7200/0xdf5a7ac59f174d24 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730159:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7600/0xdf5a7ac59f174d2b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730163:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7600/0xdf5a7ac59f174d2b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730167:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7600/0xdf5a7ac59f174d2b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730174:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7fc7600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762789.730176:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7600/0xdf5a7ac59f174d2b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730180:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7600/0xdf5a7ac59f174d2b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730186:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba600) 00010000:00010000:3.0:1789762789.730188:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762789.730191:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7600/0xdf5a7ac59f174d2b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730197:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7600/0xdf5a7ac59f174d2b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730226:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc7000/0xdf5a7ac59f174d1d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c57d expref: 9 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730241:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6e00/0xdf5a7ac59f174d32 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c57d expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.730249:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125ac00) 00010000:00010000:3.0:1789762789.730251:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762789.730254:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7fc6e00, rc 0) 00000100:00100000:3.0:1789762789.730298:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13302:x1876702067311616:12345-192.168.203.37@tcp:101:dd.500 Request processed in 558us (659us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762789.730307:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 174 00000100:00100000:2.0:1789762789.731242:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762789.731274:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067311744 00000100:00100000:3.0:1789762789.731283:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067311744 00000100:00100000:3.0:1789762789.731308:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 175 00000100:00100000:3.0:1789762789.731316:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:13302:x1876702067311744:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762789.731330:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762789.731340:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7fc6800/0xdf5a7ac59f174d39 lrc: 2/0,0 mode: --/CR res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c584 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.731435:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7fc7400/0xdf5a7ac59f174d40 lrc: 3/1,0 mode: --/PR res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.731443:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7fc7400/0xdf5a7ac59f174d40 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.731450:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7fc7400/0xdf5a7ac59f174d40 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762789.731467:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xb:0x0]: returning size 16384 00000004:00200000:3.0:1789762789.731470:0:11692:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000100:00100000:0.0:1789762789.731496:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:3.0:1789762789.731497:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7fc6800/0xdf5a7ac59f174d39 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c584 expref: 15 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.731516:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7fc7400/0xdf5a7ac59f174d40 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c584 expref: 14 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.731526:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:3.0:1789762789.731564:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:3.0:1789762789.731567:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7fc7400, rc 0) 00000100:00100000:1.0:1789762789.731574:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067311872 00000100:00100000:1.0:1789762789.731580:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067311872 00000100:00100000:3.0:1789762789.731597:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13302:x1876702067311744:12345-192.168.203.37@tcp:101:dd.500 Request processed in 282us (355us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762789.731615:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 175 00000100:00100000:1.0:1789762789.731620:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 23 00000100:00100000:1.0:1789762789.731630:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067311872:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762789.731640:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762789.731653:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6e00/0xdf5a7ac59f174d32 lrc: 0/0,0 mode: --/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c57d expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762789.731665:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125ac00) 00010000:00010000:1.0:1789762789.731668:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762789.731693:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067311872:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 67us (204us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762789.731699:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 23 00000100:00100000:0.0:1789762789.732666:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762789.732713:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067312128 00000100:00100000:2.0:1789762789.732718:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067312128 00000100:00100000:2.0:1789762789.732749:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 176 00000100:00100000:2.0:1789762789.732755:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb930700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13302:x1876702067312128:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:2.0:1789762789.732763:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762789.732773:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174d47 lrc: 2/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c58b expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.732830:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d4e lrc: 3/0,1 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.732847:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d4e lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.732854:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d4e lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.732863:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10800/0xdf5a7ac59f174d55 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xb:0x0].0x73550ecb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.732868:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10800/0xdf5a7ac59f174d55 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.732880:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10800/0xdf5a7ac59f174d55 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762789.733025:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee8c1500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074647936:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762789.733061:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762789.733083:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f61c8f00 going to sleep for 11 seconds 00000100:00100000:0.0:1789762789.733097:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074647936 00000100:00100000:0.0:1789762789.733108:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074647936 00000100:00100000:0.0:1789762789.733135:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:0.0:1789762789.733141:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3befb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074647936:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762789.733196:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3befb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074647936:12345-0@lo:1000:mdt00_005.0 Request processed in 55us (134us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762789.733202:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00100000:2.0:1789762789.733218:0:13389:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967365 gen 2 00000004:00080000:2.0:1789762789.733223:0:13389:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967365 committed transno 4294967365 00000100:00100000:2.0:1789762789.733243:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee8c1500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074647936:0@lo:1000:mdt00_005.0 00000004:00080000:2.0:1789762789.733481:0:13389:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x6:0x0], index 0: 0 00000004:00080000:2.0:1789762789.733488:0:13389:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x6:0x0], index 1: 0 00010000:00010000:2.0:1789762789.733621:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11600/0xdf5a7ac59f174d5c lrc: 3/1,0 mode: --/PR res: [0x240000402:0xc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733631:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11600/0xdf5a7ac59f174d5c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733640:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11600/0xdf5a7ac59f174d5c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733669:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d4e lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733675:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d4e lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733679:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d4e lrc: 3/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733691:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d4e lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733695:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762789.733697:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d4e lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733702:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d4e lrc: 5/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733708:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:2.0:1789762789.733710:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762789.733713:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d4e lrc: 3/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733717:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174d4e lrc: 0/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762789.733725:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003d357930 reply state 00000000b5592f15 transno 4294967364 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10800/0xdf5a7ac59f174d55 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733732:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10800/0xdf5a7ac59f174d55 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733736:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x73550ecb (ffff9c37d3bee200) 00010000:00010000:2.0:1789762789.733738:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762789.733740:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762789.733756:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174d47 lrc: 0/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c58b expref: 17 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733765:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11600/0xdf5a7ac59f174d5c lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c58b expref: 16 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762789.733775:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xc:0x0].0x0 (ffff9c37d3beef00) 00010000:00010000:2.0:1789762789.733776:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762789.733778:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b11600, rc 0) 00000100:00100000:2.0:1789762789.733806:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb930700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:13302:x1876702067312128:12345-192.168.203.37@tcp:101:dd.500 Request processed in 1051us (1141us total) trans 4294967364 rc 0/0 00000100:00100000:2.0:1789762789.733811:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 176 00000100:00100000:0.0:1789762789.738022:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762789.738054:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067312256 00000100:00100000:2.0:1789762789.738060:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067312256 00000100:00100000:2.0:1789762789.738088:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 103 00000100:00100000:2.0:1789762789.738094:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0+-99:2394:x1876702067312256:12345-192.168.203.37@tcp:8:dd.500 00010000:00080000:2.0:1789762789.738113:0:9037:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0000: connection from fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp t0 exp 0000000000000000 cur 262 last 0 00000020:00000080:2.0:1789762789.738135:0:9037:0:(genops.c:1385:class_connect()) connect: client fa6fc127-0d58-47ca-82d1-2c8ad7b086c1, cookie 0xdf5a7ac59f174d6a 00002000:00100000:2.0:1789762789.738145:0:9037:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0000: cli fa6fc127-0d58-47ca-82d1-2c8ad7b086c1/ffff9c36cbb22000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 201064448 ocd_index: 0 ocd_group 0 00002000:00100000:2.0:1789762789.738153:0:9037:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0000: cli supports cksum type f7, return f7 00002000:00080000:2.0:1789762789.738260:0:9037:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0000: get connection from MDS 0 00000100:00080000:2.0:1789762789.738280:0:9037:0:(import.c:65:import_set_state_nolock()) ffff9c36cbb21000 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789762789.738312:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2394:x1876702067312256:12345-192.168.203.37@tcp:8:dd.500 Request processed in 218us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762789.738318:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 103 00000100:00100000:0.0:1789762789.739162:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762789.739194:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067312384 00000100:00100000:3.0:1789762789.739199:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067312384 00000100:00080000:1.0:1789762789.739237:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762789.739238:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 104 00000100:00100000:3.0:1789762789.739247:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13302:x1876702067312384:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762789.739255:0:8052:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762789.739266:0:8052:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000682c79ca/0xdf5a7ac59f174d71 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c592 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762789.739282:0:8052:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x6:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762789.739293:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x6:0x0].0x0 since=262s v=(0 0) 00010000:00010000:3.0:1789762789.739297:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762789.739304:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000682c79ca/0xdf5a7ac59f174d71 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c592 expref: 7 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.739314:0:8052:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000682c79ca/0xdf5a7ac59f174d71 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c592 expref: 7 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762789.739322:0:8052:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7fc7000, rc 0) 00000100:00100000:3.0:1789762789.739348:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13302:x1876702067312384:12345-192.168.203.37@tcp:101:dd.500 Request processed in 103us (187us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762789.739353:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 104 00000100:00100000:2.0:1789762789.746631:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762789.746667:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067312512 00000100:00100000:3.0:1789762789.746673:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067312512 00002000:00100000:3.0:1789762789.746697:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x6:0x0] (0->1048575) req@ffff9c37e3bef480 x1876702067312512/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762789.746716:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bef480 with x1876702067312512 ext(0->1048575) 00002000:00010000:3.0:1789762789.746720:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e3bef480 00000100:00100000:3.0:1789762789.746739:0:8059:0:(nrs_tbf.c:1585:nrs_tbf_cli_findadd()) unknown.0 NRS: TBF class insert: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 3 deadline 0 time 262993813440 state 0x0 ref 1 00000100:00100000:3.0:1789762789.746747:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 3 deadline 263043813440 time 262993813440 state 0x1 ref 1 00000100:00100000:3.0:1789762789.746768:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 2 deadline 263043813440 time 262993846665 state 0x0 ref 1 00000100:00100000:3.0:1789762789.746777:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067312512:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762789.755872:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.755909:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067312640 00000100:00100000:1.0:1789762789.755951:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067312640 00002000:00100000:1.0:1789762789.755980:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0x6:0x0] (1048576->2097151) req@ffff9c37ee8c1500 x1876702067312640/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762789.755990:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee8c1500 with x1876702067312640 ext(1048576->2097151) 00002000:00010000:1.0:1789762789.755994:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ee8c1500 00000100:00100000:1.0:1789762789.756006:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 2 deadline 263043846665 time 262993846665 state 0x1 ref 2 00000100:00100000:1.0:1789762789.756032:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 1 deadline 263043846665 time 263003110532 state 0x0 ref 2 00000100:00100000:1.0:1789762789.756045:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee8c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2397:x1876702067312640:12345-192.168.203.37@tcp:4:dd.500 00000001:00100000:3.0:1789762789.762423:0:8082:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000001:00080000:0.0:1789762789.776906:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967557 is committed 00000100:00100000:1.0:1789762789.777028:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067312512:12345-192.168.203.37@tcp:4:dd.500 Request processed in 30244us (30393us total) trans 4294967556 rc 0/0 00000100:00100000:3.0:1789762789.777028:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee8c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067312640:12345-192.168.203.37@tcp:4:dd.500 Request processed in 20976us (21148us total) trans 4294967557 rc 0/0 00000100:00100000:1.0:1789762789.777046:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 0 00000100:00100000:3.0:1789762789.777055:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1 00002000:00100000:1.0:1789762789.777060:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x6:0x0] (0->1048575) req@ffff9c37e3bef480 x1876702067312512/t4294967556(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762789.777078:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bef480 with x1876702067312512 ext(0->1048575) 00000100:00100000:3.0:1789762789.777080:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 1 deadline 263043846665 time 263003110532 state 0x2 ref 0 00002000:00010000:1.0:1789762789.777084:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e3bef480 00002000:00100000:3.0:1789762789.777094:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0x6:0x0] (1048576->2097151) req@ffff9c37ee8c1500 x1876702067312640/t4294967557(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762789.777102:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee8c1500 with x1876702067312640 ext(1048576->2097151) 00002000:00010000:3.0:1789762789.777106:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ee8c1500 00000100:00100000:0.0:1789762789.780371:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.780426:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067312768 00000100:00100000:1.0:1789762789.780437:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067312768 00002000:00100000:1.0:1789762789.780470:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x6:0x0] (2097152->3145727) req@ffff9c37ff6b9500 x1876702067312768/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762789.780478:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff6b9500 with x1876702067312768 ext(2097152->3145727) 00002000:00010000:1.0:1789762789.780481:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff6b9500 00000100:00100000:1.0:1789762789.780498:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 1 deadline 263043846665 time 263003110532 state 0x0 ref 1 00000100:00100000:1.0:1789762789.780508:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 1 deadline 263053110532 time 263003110532 state 0x1 ref 1 00000100:00100000:1.0:1789762789.780532:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263053110532 time 263027609893 state 0x0 ref 1 00000100:00100000:1.0:1789762789.780542:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067312768:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762789.782958:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067312768:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2378us (2549us total) trans 4294967558 rc 0/0 00000100:00100000:1.0:1789762789.782965:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 2 00000100:00100000:1.0:1789762789.782971:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263053110532 time 263027609893 state 0x2 ref 0 00002000:00100000:1.0:1789762789.782981:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x6:0x0] (2097152->3145727) req@ffff9c37ff6b9500 x1876702067312768/t4294967558(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762789.782987:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff6b9500 with x1876702067312768 ext(2097152->3145727) 00002000:00010000:1.0:1789762789.782990:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff6b9500 00000100:00100000:2.0:1789762789.787565:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.787647:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067312896 00000100:00100000:1.0:1789762789.787664:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067312896 00002000:00100000:1.0:1789762789.787712:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0x6:0x0] (3145728->4194303) req@ffff9c36d19f7800 x1876702067312896/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762789.787722:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d19f7800 with x1876702067312896 ext(3145728->4194303) 00002000:00010000:1.0:1789762789.787725:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36d19f7800 00000100:00100000:1.0:1789762789.787736:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263053110532 time 263027609893 state 0x0 ref 1 00000100:00100000:1.0:1789762789.787770:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263077609893 time 263027609893 state 0x1 ref 1 00000100:00100000:0.0:1789762789.788028:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762789.788078:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067313024 00000100:00100000:3.0:1789762789.788083:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067313024 00000100:00100000:3.0:1789762789.788106:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 105 00000100:00100000:3.0:1789762789.788111:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0+-99:2394:x1876702067313024:12345-192.168.203.37@tcp:8:dd.500 00010000:00080000:3.0:1789762789.788142:0:8054:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp t0 exp 0000000000000000 cur 263 last 0 00000020:00000080:3.0:1789762789.788164:0:8054:0:(genops.c:1385:class_connect()) connect: client fa6fc127-0d58-47ca-82d1-2c8ad7b086c1, cookie 0xdf5a7ac59f174d78 00002000:00100000:3.0:1789762789.788173:0:8054:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli fa6fc127-0d58-47ca-82d1-2c8ad7b086c1/ffff9c37c816a800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:3.0:1789762789.788187:0:8054:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:3.0:1789762789.788324:0:8054:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:3.0:1789762789.788345:0:8054:0:(import.c:65:import_set_state_nolock()) ffff9c37c8169000 : changing import state from RECOVER to FULL 00000100:00100000:3.0:1789762789.788389:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2394:x1876702067313024:12345-192.168.203.37@tcp:8:dd.500 Request processed in 277us (362us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762789.788395:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 105 00000100:00100000:2.0:1789762789.789282:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762789.789356:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067313152 00000100:00100000:0.0:1789762789.789362:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067313152 00000100:00100000:0.0:1789762789.789401:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 106 00000100:00100000:0.0:1789762789.789408:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13302:x1876702067313152:12345-192.168.203.37@tcp:101:dd.500 00000100:00080000:1.0:1789762789.789414:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1789762789.789418:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762789.789434:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174d7f lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c599 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762789.789454:0:9037:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x6:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789762789.789468:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x6:0x0].0x0 since=263s v=(0 0) 00010000:00010000:0.0:1789762789.789472:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762789.789479:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174d7f lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c599 expref: 7 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.789492:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174d7f lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c599 expref: 7 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762789.789512:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa7200, rc 0) 00000100:00100000:0.0:1789762789.789540:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13302:x1876702067313152:12345-192.168.203.37@tcp:101:dd.500 Request processed in 132us (274us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762789.789544:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 106 00000100:00100000:0.0:1789762789.801396:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762789.801442:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067313280 00000100:00100000:2.0:1789762789.801455:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067313280 00002000:00100000:2.0:1789762789.801484:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x6:0x0] (0->1048575) req@ffff9c37ca32b800 x1876702067313280/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762789.801495:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ca32b800 with x1876702067313280 ext(0->1048575) 00002000:00010000:2.0:1789762789.801500:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ca32b800 00000100:00100000:2.0:1789762789.801517:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263077609893 time 263027609893 state 0x1 ref 2 00000100:00100000:2.0:1789762789.808434:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762789.808511:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067313408 00000100:00100000:3.0:1789762789.808517:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067313408 00002000:00100000:3.0:1789762789.808537:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0x6:0x0] (1048576->2097151) req@ffff9c37e3b95180 x1876702067313408/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762789.808544:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b95180 with x1876702067313408 ext(1048576->2097151) 00002000:00010000:3.0:1789762789.808547:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3b95180 00000100:00100000:3.0:1789762789.808556:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263077609893 time 263027609893 state 0x1 ref 3 00000100:00100000:1.0:1789762789.830581:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263127652562 time 263077652562 state 0x1 ref 3 00000100:00100000:1.0:1789762789.830597:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d19f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067312896:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762789.833660:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d19f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2395:x1876702067312896:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3065us (46095us total) trans 4294967559 rc 0/0 00000100:00100000:1.0:1789762789.833667:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 3 00002000:00100000:1.0:1789762789.833675:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x6:0x0] (3145728->4194303) req@ffff9c36d19f7800 x1876702067312896/t4294967559(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762789.833682:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d19f7800 with x1876702067312896 ext(3145728->4194303) 00002000:00010000:1.0:1789762789.833687:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36d19f7800 00000100:00100000:1.0:1789762789.880624:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263177693166 time 263127693166 state 0x1 ref 2 00000100:00100000:1.0:1789762789.880641:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ca32b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067313280:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762789.888425:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3b94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074648064:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762789.888491:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762789.888561:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074648064 00000100:00100000:2.0:1789762789.888567:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074648064 00000100:00100000:2.0:1789762789.888616:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:2.0:1789762789.888629:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074648064:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762789.888659:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762789.888711:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074648064:12345-0@lo:41:osp-pre-1-0.0 Request processed in 86us (221us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762789.888718:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000100:00100000:0.0:1789762789.888744:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3b94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074648064:0@lo:41:osp-pre-1-0.0 00000001:00100000:3.0:1789762789.891634:0:9000:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000001:00080000:1.0:1789762789.902510:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967346 is committed 00000100:00100000:0.0:1789762789.902596:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ca32b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067313280:12345-192.168.203.37@tcp:4:dd.500 Request processed in 21955us (101197us total) trans 4294967346 rc 0/0 00000100:00100000:0.0:1789762789.902610:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 4 00002000:00100000:0.0:1789762789.902625:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0x6:0x0] (0->1048575) req@ffff9c37ca32b800 x1876702067313280/t4294967346(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762789.902637:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ca32b800 with x1876702067313280 ext(0->1048575) 00002000:00010000:0.0:1789762789.902643:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ca32b800 00000100:00100000:0.0:1789762789.905540:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762789.905579:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067313536 00000100:00100000:3.0:1789762789.905587:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067313536 00002000:00100000:3.0:1789762789.905618:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x6:0x0] (2097152->3145727) req@ffff9c37ca32ad80 x1876702067313536/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762789.905635:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ca32ad80 with x1876702067313536 ext(2097152->3145727) 00002000:00010000:3.0:1789762789.905639:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ca32ad80 00000100:00100000:3.0:1789762789.905660:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263177693166 time 263127693166 state 0x1 ref 2 00000100:00100000:3.0:1789762789.915443:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.915511:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067313664 00000100:00100000:1.0:1789762789.915520:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067313664 00002000:00100000:1.0:1789762789.915568:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x6:0x0] (3145728->4194303) req@ffff9c37d8c8f800 x1876702067313664/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762789.915580:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c8f800 with x1876702067313664 ext(3145728->4194303) 00002000:00010000:1.0:1789762789.915595:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8c8f800 00000100:00100000:1.0:1789762789.915619:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263177693166 time 263127693166 state 0x1 ref 3 00000100:00100000:2.0:1789762789.922876:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762789.922956:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067313792 00000100:00100000:0.0:1789762789.922965:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067313792 00002000:00100000:0.0:1789762789.923002:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x6:0x0] (4194304->5242879) req@ffff9c37ca314000 x1876702067313792/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762789.923012:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ca314000 with x1876702067313792 ext(4194304->5242879) 00002000:00010000:0.0:1789762789.923015:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ca314000 00000100:00100000:0.0:1789762789.923030:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263177693166 time 263127693166 state 0x1 ref 4 00000100:00100000:0.0:1789762789.930046:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.930115:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067313920 00000100:00100000:1.0:1789762789.930121:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067313920 00002000:00100000:1.0:1789762789.930149:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0x6:0x0] (5242880->6291455) req@ffff9c37e3b94000 x1876702067313920/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762789.930158:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b94000 with x1876702067313920 ext(5242880->6291455) 00002000:00010000:1.0:1789762789.930161:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e3b94000 00000100:00100000:1.0:1789762789.930173:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263177693166 time 263127693166 state 0x1 ref 5 00000100:00100000:2.0:1789762789.930715:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263227787533 time 263177787533 state 0x1 ref 5 00000100:00100000:2.0:1789762789.930730:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067313408:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762789.933742:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2398:x1876702067313408:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3014us (125306us total) trans 4294967347 rc 0/0 00000100:00100000:0.0:1789762789.933751:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 5 00002000:00100000:0.0:1789762789.933759:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x6:0x0] (1048576->2097151) req@ffff9c37e3b95180 x1876702067313408/t4294967347(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762789.933766:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b95180 with x1876702067313408 ext(1048576->2097151) 00002000:00010000:0.0:1789762789.933770:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3b95180 00000100:00100000:2.0:1789762789.938415:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.938460:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067314048 00000100:00100000:1.0:1789762789.938465:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067314048 00002000:00100000:1.0:1789762789.938484:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x6:0x0] (6291456->7340031) req@ffff9c37d184c700 x1876702067314048/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762789.938492:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184c700 with x1876702067314048 ext(6291456->7340031) 00002000:00010000:1.0:1789762789.938495:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184c700 00000100:00100000:1.0:1789762789.938504:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263227787533 time 263177787533 state 0x1 ref 5 00000100:00100000:3.0:1789762789.945307:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762789.945356:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067314176 00000100:00100000:0.0:1789762789.945362:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067314176 00002000:00100000:0.0:1789762789.945386:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x6:0x0] (7340032->8388607) req@ffff9c37d8c8d880 x1876702067314176/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762789.945393:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c8d880 with x1876702067314176 ext(7340032->8388607) 00002000:00010000:0.0:1789762789.945397:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8c8d880 00000100:00100000:0.0:1789762789.945406:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263227787533 time 263177787533 state 0x1 ref 6 00000100:00100000:2.0:1789762789.952010:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.952061:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067314304 00000100:00100000:1.0:1789762789.952067:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067314304 00002000:00100000:1.0:1789762789.952089:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0x6:0x0] (4194304->5242879) req@ffff9c37d184ce00 x1876702067314304/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762789.952097:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184ce00 with x1876702067314304 ext(4194304->5242879) 00002000:00010000:1.0:1789762789.952101:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184ce00 00000100:00100000:1.0:1789762789.952120:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263227787533 time 263177787533 state 0x1 ref 7 00000100:00100000:2.0:1789762789.958763:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762789.958855:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067314432 00000100:00100000:0.0:1789762789.958861:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067314432 00002000:00100000:0.0:1789762789.958882:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x6:0x0] (5242880->6291455) req@ffff9c37d184ea00 x1876702067314432/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762789.958889:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184ea00 with x1876702067314432 ext(5242880->6291455) 00002000:00010000:0.0:1789762789.958892:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184ea00 00000100:00100000:0.0:1789762789.958901:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263227787533 time 263177787533 state 0x1 ref 8 00000100:00100000:3.0:1789762789.980808:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263277875226 time 263227875226 state 0x1 ref 8 00000100:00100000:3.0:1789762789.980827:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ca32ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:2398:x1876702067313536:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762789.983677:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ca32ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:2398:x1876702067313536:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2855us (78137us total) trans 4294967348 rc 0/0 00000100:00100000:3.0:1789762789.983685:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 6 00002000:00100000:3.0:1789762789.983692:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0x6:0x0] (2097152->3145727) req@ffff9c37ca32ad80 x1876702067313536/t4294967348(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762789.983700:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ca32ad80 with x1876702067313536 ext(2097152->3145727) 00002000:00010000:3.0:1789762789.983704:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ca32ad80 00000100:00100000:2.0:1789762789.986037:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762789.986080:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067314560 00000100:00100000:0.0:1789762789.986087:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067314560 00002000:00100000:0.0:1789762789.986119:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0x6:0x0] (6291456->7340031) req@ffff9c37e3bc7100 x1876702067314560/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762789.986126:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bc7100 with x1876702067314560 ext(6291456->7340031) 00002000:00010000:0.0:1789762789.986130:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3bc7100 00000100:00100000:0.0:1789762789.986141:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263277875226 time 263227875226 state 0x1 ref 8 00000100:00100000:0.0:1789762789.996631:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762789.996682:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067314944 00000100:00100000:2.0:1789762789.996688:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067314944 00000100:00100000:2.0:1789762789.996729:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 42 00000100:00100000:2.0:1789762789.996738:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:13302:x1876702067314944:12345-192.168.203.37@tcp:35:dd.500 00000100:00100000:0.0:1789762789.996750:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762789.996778:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067314688 00000100:00100000:1.0:1789762789.996784:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067314688 00002000:00100000:1.0:1789762789.996808:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0x6:0x0] (7340032->8388607) req@ffff9c37fd9c3800 x1876702067314688/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/0 e 0 to 0 dl 1789762800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762789.996816:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fd9c3800 with x1876702067314688 ext(7340032->8388607) 00002000:00010000:1.0:1789762789.996820:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fd9c3800 00000100:00100000:1.0:1789762789.996838:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263277875226 time 263227875226 state 0x1 ref 9 00000100:00100000:2.0:1789762789.997010:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13302:x1876702067314944:12345-192.168.203.37@tcp:35:dd.500 Request processed in 273us (381us total) trans 4294967365 rc 0/0 00000100:00100000:2.0:1789762789.997015:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 42 00010000:00010000:1.0:1789762790.016079:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762790.030907:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263327972609 time 263277972609 state 0x1 ref 9 00000100:00100000:0.0:1789762790.030959:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:2398:x1876702067313664:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762790.033656:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+16:2398:x1876702067313664:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2703us (118221us total) trans 4294967349 rc 0/0 00000100:00100000:3.0:1789762790.033663:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 7 00002000:00100000:3.0:1789762790.033672:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x6:0x0] (3145728->4194303) req@ffff9c37d8c8f800 x1876702067313664/t4294967349(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762790.033680:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c8f800 with x1876702067313664 ext(3145728->4194303) 00002000:00010000:3.0:1789762790.033685:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8c8f800 00000100:00100000:2.0:1789762790.037427:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.037516:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067315200 00000100:00100000:0.0:1789762790.037524:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067315200 00000100:00100000:0.0:1789762790.037564:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 177 00000100:00100000:0.0:1789762790.037572:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13314:x1876702067315200:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762790.037587:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762790.037601:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174d86 lrc: 2/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5a0 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.037683:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174d8d lrc: 3/0,1 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.037696:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174d8d lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.037705:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174d8d lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.037724:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7400/0xdf5a7ac59f174d94 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.037729:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7400/0xdf5a7ac59f174d94 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.037734:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7400/0xdf5a7ac59f174d94 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762790.038000:0:6418:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x7:0x0], index 1: 0 00000004:00080000:0.0:1789762790.038005:0:6418:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x7:0x0], index 0: 0 00010000:00010000:0.0:1789762790.038115:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174d9b lrc: 3/1,0 mode: --/PR res: [0x240000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038121:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174d9b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038127:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174d9b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038149:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174d8d lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038154:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174d8d lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038158:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174d8d lrc: 3/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038163:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174d8d lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038177:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762790.038181:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174d8d lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038186:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174d8d lrc: 5/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038196:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:0.0:1789762790.038198:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762790.038201:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174d8d lrc: 3/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038205:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174d8d lrc: 0/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762790.038215:0:6418:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000030606b5b reply state 00000000eda96974 transno 4294967366 ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7400/0xdf5a7ac59f174d94 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038222:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7400/0xdf5a7ac59f174d94 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038226:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x2b0e463d (ffff9c37e125aa00) 00010000:00010000:0.0:1789762790.038228:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762790.038230:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762790.038249:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174d86 lrc: 0/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a0 expref: 21 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038258:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174d9b lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a0 expref: 20 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.038271:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xd:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762790.038272:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762790.038274:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa6200, rc 0) 00000100:00100000:0.0:1789762790.038304:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:13314:x1876702067315200:12345-192.168.203.37@tcp:101:dd.0 Request processed in 733us (879us total) trans 4294967366 rc 0/0 00000100:00100000:0.0:1789762790.038309:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 177 00000100:00100000:0.0:1789762790.045242:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.045286:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067315328 00000100:00100000:3.0:1789762790.045292:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067315328 00000100:00100000:3.0:1789762790.045328:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 107 00000100:00100000:3.0:1789762790.045414:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+16:13315:x1876702067315328:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762790.045422:0:8052:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762790.045433:0:8052:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5a7 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762790.045463:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00002000:00010000:3.0:1789762790.045482:0:8052:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x7:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762790.045497:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=263s v=(0 0) 00000100:00100000:0.0:1789762790.045501:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067315456 00010000:00010000:3.0:1789762790.045502:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(0 0) 00000100:00100000:0.0:1789762790.045507:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067315456 00010000:00010000:3.0:1789762790.045513:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 18 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.045525:0:8052:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 18 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.045533:0:8052:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7fc6800, rc 0) 00000100:00100000:0.0:1789762790.045539:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 108 00000100:00100000:0.0:1789762790.045549:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13313:x1876702067315456:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762790.045557:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762790.045566:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000021769b91/0xdf5a7ac59f174db0 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5ae expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762790.045569:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13315:x1876702067315328:12345-192.168.203.37@tcp:101:dd.0 Request processed in 224us (325us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762790.045585:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 107 00010000:00010000:0.0:1789762790.045595:0:8054:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 19 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.045602:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762790.045605:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762790.045610:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000021769b91/0xdf5a7ac59f174db0 lrc: 4/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ae expref: 18 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.045638:0:8054:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 18 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.045659:0:8054:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 18 pid: 8052 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.045670:0:8054:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7fb9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8054:1876702074648192:192.168.203.37@tcp:104: 00000100:00100000:0.0:1789762790.045732:0:8054:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f61c8780 going to sleep for 11 seconds 00000100:00100000:2.0:1789762790.045909:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.045956:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067315584 00000100:00100000:0.0:1789762790.045961:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067315584 00000100:00100000:0.0:1789762790.045985:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 109 00000100:00100000:0.0:1789762790.045990:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:13314:x1876702067315584:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762790.045997:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762790.046003:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005a11c428/0xdf5a7ac59f174db7 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5b5 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.046015:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762790.046018:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(2 0) 00010000:00010000:0.0:1789762790.046021:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005a11c428/0xdf5a7ac59f174db7 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5b5 expref: 20 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.046028:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005a11c428/0xdf5a7ac59f174db7 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5b5 expref: 20 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.046035:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa7600, rc 0) 00000100:00100000:0.0:1789762790.046058:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:13314:x1876702067315584:12345-192.168.203.37@tcp:101:dd.0 Request processed in 68us (150us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762790.046073:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 109 00000100:00100000:0.0:1789762790.046714:0:8054:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7fb9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8054:1876702074648192:192.168.203.37@tcp:104: 00010000:00010000:0.0:1789762790.046736:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000021769b91/0xdf5a7ac59f174db0 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ae expref: 19 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.046749:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa6c00, rc 0) 00000100:00100000:0.0:1789762790.046784:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:13313:x1876702067315456:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1237us (1320us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762790.046792:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 108 00000100:00100000:2.0:1789762790.055181:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.055224:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067315712 00000100:00100000:1.0:1789762790.055231:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067315712 00000100:00100000:1.0:1789762790.055262:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 110 00000100:00100000:1.0:1789762790.055269:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13317:x1876702067315712:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762790.055277:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762790.055288:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 2/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5bc expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762790.055302:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x7:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789762790.055312:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=263s v=(0 0) 00010000:00010000:1.0:1789762790.055314:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762790.055319:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 15 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.055328:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 15 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.055343:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94c00, rc 0) 00000100:00100000:1.0:1789762790.055371:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:13317:x1876702067315712:12345-192.168.203.37@tcp:101:dd.0 Request processed in 103us (191us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.055377:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 110 00000100:00100000:2.0:1789762790.062812:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.062865:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067315968 00000100:00100000:1.0:1789762790.062873:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067315968 00000100:00100000:1.0:1789762790.062911:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 111 00000100:00100000:1.0:1789762790.062974:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:13320:x1876702067315968:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762790.062985:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762790.062996:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f174dc5 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5ca expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.063019:0:8053:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 16 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.063026:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762790.063028:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789762790.063031:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f174dc5 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ca expref: 16 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.063053:0:8053:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 16 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.063061:0:8053:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 16 pid: 9038 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762790.063081:0:8053:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fdfe0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074648320:192.168.203.37@tcp:104: 00000100:00100000:1.0:1789762790.063126:0:8053:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7b7ed80 going to sleep for 11 seconds 00000100:00100000:1.0:1789762790.063967:0:8053:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fdfe0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074648320:192.168.203.37@tcp:104: 00010000:00010000:1.0:1789762790.063982:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f174dc5 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ca expref: 16 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.063991:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94a00, rc 0) 00000100:00100000:1.0:1789762790.064015:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+16:13320:x1876702067315968:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1097us (1204us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.064021:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 111 00000100:00100000:2.0:1789762790.065243:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.065313:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067316096 00000100:00100000:0.0:1789762790.065319:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067316096 00000100:00100000:0.0:1789762790.065347:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 112 00000100:00100000:0.0:1789762790.065354:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+16:13325:x1876702067316096:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762790.065361:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762790.065369:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174dcc lrc: 2/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5d1 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.065382:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762790.065386:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(2 0) 00010000:00010000:0.0:1789762790.065389:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174dcc lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 4 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5d1 expref: 17 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.065397:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174dcc lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 4 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5d1 expref: 17 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.065415:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa6600, rc 0) 00000100:00100000:0.0:1789762790.065442:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:13325:x1876702067316096:12345-192.168.203.37@tcp:101:dd.0 Request processed in 88us (200us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762790.065447:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 112 00000100:00100000:2.0:1789762790.066842:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.066863:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.066888:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067316224 00000100:00100000:1.0:1789762790.066893:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067316224 00000100:00100000:2.0:1789762790.066955:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067315840 00000100:00100000:1.0:1789762790.066960:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 113 00000100:00100000:2.0:1789762790.066960:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067315840 00000100:00100000:1.0:1789762790.066974:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:13322:x1876702067316224:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762790.066983:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762790.066993:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f174dd3 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5d8 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762790.067002:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 114 00000100:00100000:2.0:1789762790.067014:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:13318:x1876702067315840:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762790.067018:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(2 0) 00010000:00010000:1.0:1789762790.067023:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762790.067025:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762790.067030:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f174dd3 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 5 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5d8 expref: 20 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.067035:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f174dda lrc: 2/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5c3 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.067042:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f174dd3 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 5 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5d8 expref: 20 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.067054:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x7:0x0].0x0 since=0s v=(2 0) 00010000:00010000:1.0:1789762790.067058:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a95a00, rc 0) 00010000:00010000:2.0:1789762790.067059:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762790.067066:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f174dda lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5c3 expref: 18 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.067084:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f174dda lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5c3 expref: 18 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.067093:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b11c00, rc 0) 00000100:00100000:1.0:1789762790.067095:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:13322:x1876702067316224:12345-192.168.203.37@tcp:101:dd.0 Request processed in 122us (251us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.067103:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 113 00000100:00100000:2.0:1789762790.067123:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13318:x1876702067315840:12345-192.168.203.37@tcp:101:dd.0 Request processed in 110us (256us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762790.067131:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 114 00000100:00100000:0.0:1789762790.067138:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.067160:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067316352 00000100:00100000:2.0:1789762790.067164:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067316352 00000100:00100000:2.0:1789762790.067194:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 115 00000100:00100000:2.0:1789762790.067200:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13319:x1876702067316352:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762790.067206:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762790.067213:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174de1 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5df expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.067223:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x7:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762790.067226:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762790.067230:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174de1 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5df expref: 19 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.067236:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174de1 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5df expref: 19 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.067243:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b11800, rc 0) 00000100:00100000:2.0:1789762790.067264:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:13319:x1876702067316352:12345-192.168.203.37@tcp:101:dd.0 Request processed in 65us (133us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762790.067274:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 115 00000100:00100000:3.0:1789762790.070892:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.070943:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067316480 00000100:00100000:1.0:1789762790.070948:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067316480 00002000:00100000:1.0:1789762790.070968:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x7:0x0] (2097152->3145727) req@ffff9c37d8c8c700 x1876702067316480/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762790.070976:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c8c700 with x1876702067316480 ext(2097152->3145727) 00010000:00010000:1.0:1789762790.070981:0:8059:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 20 pid: 8052 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.070990:0:8059:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 20 pid: 8052 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762790.070995:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9c37d8c8c700 00000100:00100000:1.0:1789762790.071012:0:8059:0:(nrs_tbf.c:1585:nrs_tbf_cli_findadd()) unknown.0 NRS: TBF class insert: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 3 deadline 0 time 263318088200 state 0x0 ref 1 00000100:00100000:1.0:1789762790.071021:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 3 deadline 263318188200 time 263318088200 state 0x1 ref 1 00000100:00100000:1.0:1789762790.071041:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 263318188200 time 263318119682 state 0x0 ref 1 00000100:00100000:1.0:1789762790.071049:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:2398:x1876702067316480:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762790.073065:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.073104:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067316608 00000100:00100000:0.0:1789762790.073110:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067316608 00000100:00100000:0.0:1789762790.073144:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 116 00000100:00100000:0.0:1789762790.073151:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:13323:x1876702067316608:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762790.073159:0:8052:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762790.073169:0:8052:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000684da15/0xdf5a7ac59f174de8 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5e6 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.073184:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(3 0) 00010000:00010000:0.0:1789762790.073187:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(4 0) 00010000:00010000:0.0:1789762790.073192:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000684da15/0xdf5a7ac59f174de8 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5e6 expref: 23 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.073201:0:8052:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000684da15/0xdf5a7ac59f174de8 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5e6 expref: 23 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.073209:0:8052:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa6e00, rc 0) 00000100:00100000:0.0:1789762790.073240:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:13323:x1876702067316608:12345-192.168.203.37@tcp:101:dd.0 Request processed in 90us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762790.073247:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 116 00000100:00100000:2.0:1789762790.074747:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.074808:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067316736 00000100:00100000:1.0:1789762790.074815:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067316736 00000100:00100000:1.0:1789762790.074859:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 117 00000100:00100000:1.0:1789762790.074869:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:13316:x1876702067316736:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762790.074877:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762790.074887:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000258fc572/0xdf5a7ac59f174def lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5ed expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.074901:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(4 0) 00010000:00010000:1.0:1789762790.074905:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(5 0) 00010000:00010000:1.0:1789762790.074910:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000258fc572/0xdf5a7ac59f174def lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ed expref: 24 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.074954:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000258fc572/0xdf5a7ac59f174def lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ed expref: 24 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.074968:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a95000, rc 0) 00000100:00100000:1.0:1789762790.075011:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:13316:x1876702067316736:12345-192.168.203.37@tcp:101:dd.0 Request processed in 143us (265us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.075021:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 117 00000100:00100000:2.0:1789762790.075058:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.075098:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067316864 00000100:00100000:1.0:1789762790.075104:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067316864 00000100:00100000:1.0:1789762790.075149:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 118 00000100:00100000:1.0:1789762790.075162:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:13324:x1876702067316864:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762790.075172:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762790.075182:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000c63f95b/0xdf5a7ac59f174df6 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5f4 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.075208:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(5 0) 00010000:00010000:1.0:1789762790.075212:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762790.075219:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000c63f95b/0xdf5a7ac59f174df6 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 8 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5f4 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.075233:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000c63f95b/0xdf5a7ac59f174df6 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 8 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5f4 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.075252:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94400, rc 0) 00000100:00100000:1.0:1789762790.075294:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:13324:x1876702067316864:12345-192.168.203.37@tcp:101:dd.0 Request processed in 135us (238us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.075304:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 118 00000100:00100000:3.0:1789762790.075980:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.076030:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.076034:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067316992 00000100:00100000:1.0:1789762790.076040:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067316992 00000100:00100000:2.0:1789762790.076087:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067317120 00000100:00100000:2.0:1789762790.076093:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067317120 00000100:00100000:1.0:1789762790.076110:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 119 00000100:00100000:1.0:1789762790.076124:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:13327:x1876702067316992:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762790.076134:0:8052:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1789762790.076141:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 120 00010000:00010000:1.0:1789762790.076146:0:8052:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174dfd lrc: 2/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c5fb expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762790.076154:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:13326:x1876702067317120:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762790.076169:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762790.076180:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f174e04 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c602 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.076195:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x7:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762790.076199:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(5 0) 00010000:00010000:2.0:1789762790.076204:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f174e04 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c602 expref: 22 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.076212:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f174e04 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c602 expref: 22 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.076220:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b10400, rc 0) 00000100:00100000:2.0:1789762790.076252:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:13326:x1876702067317120:12345-192.168.203.37@tcp:101:dd.0 Request processed in 102us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762790.076257:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 120 00010000:00010000:1.0:1789762790.076281:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x7:0x0].0x0 since=0s v=(5 0) 00010000:00010000:1.0:1789762790.076288:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762790.076295:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174dfd lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5fb expref: 21 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.076322:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2398:x1876702067316480:12345-192.168.203.37@tcp:4:dd.0 Request processed in 5270us (5429us total) trans 4294967350 rc 0/0 00010000:00010000:1.0:1789762790.076326:0:8052:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174dfd lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5fb expref: 21 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.076334:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 0 00000100:00100000:0.0:1789762790.076348:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 263318188200 time 263318119682 state 0x2 ref 0 00010000:00010000:1.0:1789762790.076352:0:8052:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a95c00, rc 0) 00002000:00100000:0.0:1789762790.076358:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x7:0x0] (2097152->3145727) req@ffff9c37d8c8c700 x1876702067316480/t4294967350(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762790.076367:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c8c700 with x1876702067316480 ext(2097152->3145727) 00010000:00010000:0.0:1789762790.076373:0:8059:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 25 pid: 8052 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.076382:0:8059:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 25 pid: 8052 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762790.076388:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9c37d8c8c700 00000100:00100000:1.0:1789762790.076399:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:13327:x1876702067316992:12345-192.168.203.37@tcp:101:dd.0 Request processed in 277us (420us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.076410:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 119 00000100:00100000:3.0:1789762790.077292:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.077347:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067317248 00000100:00100000:1.0:1789762790.077352:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067317248 00000100:00100000:1.0:1789762790.077385:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 121 00000100:00100000:1.0:1789762790.077395:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:13321:x1876702067317248:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762790.077405:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762790.077413:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008bd9074b/0xdf5a7ac59f174e0b lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c609 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.077430:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762790.077434:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.077440:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008bd9074b/0xdf5a7ac59f174e0b lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c609 expref: 26 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.077452:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008bd9074b/0xdf5a7ac59f174e0b lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c609 expref: 26 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.077462:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94200, rc 0) 00000100:00100000:1.0:1789762790.077496:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:13321:x1876702067317248:12345-192.168.203.37@tcp:101:dd.0 Request processed in 103us (205us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.077511:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 121 00000100:00100000:2.0:1789762790.077778:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.077805:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067317376 00000100:00100000:3.0:1789762790.077810:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067317376 00000100:00100000:3.0:1789762790.077833:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 122 00000100:00100000:3.0:1789762790.077839:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:13328:x1876702067317376:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762790.077853:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762790.077862:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cba5766/0xdf5a7ac59f174e12 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c610 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.077879:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x7:0x0].0x0 since=0s v=(6 0) 00010000:00010000:3.0:1789762790.077883:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:3.0:1789762790.077976:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cba5766/0xdf5a7ac59f174e12 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c610 expref: 22 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.077992:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001cba5766/0xdf5a7ac59f174e12 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c610 expref: 22 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.078005:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7fc7600, rc 0) 00000100:00100000:2.0:1789762790.078032:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.078035:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:13328:x1876702067317376:12345-192.168.203.37@tcp:101:dd.0 Request processed in 195us (257us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762790.078044:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 122 00000100:00100000:1.0:1789762790.078099:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067317504 00000100:00100000:1.0:1789762790.078104:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067317504 00010000:00010000:1.0:1789762790.078127:0:6405:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 26 pid: 8052 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762790.078158:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 65 00000100:00100000:1.0:1789762790.078167:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cce680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2397:x1876702067317504:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762790.078176:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1789762790.078186:0:6405:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0x7:0x0] updating lvb size from disk: 0 -> 3145728 00002000:00010000:1.0:1789762790.078189:0:6405:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0x7:0x0] updating lvb mtime from disk: 0 -> 1789762789 00002000:00010000:1.0:1789762790.078192:0:6405:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0x7:0x0] updating lvb ctime from disk: 0 -> 1789762789 00010000:00010000:1.0:1789762790.078197:0:6405:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 26 pid: 8052 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078208:0:6405:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 26 pid: 8052 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078226:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ecab1bfb/0xdf5a7ac59f174da9 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a7 expref: 26 pid: 8052 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078238:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078242:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078249:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000021769b91/0xdf5a7ac59f174db0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ae expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078273:0:6405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000021769b91/0xdf5a7ac59f174db0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ae expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078282:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078285:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078291:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005a11c428/0xdf5a7ac59f174db7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5b5 expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078300:0:6405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005a11c428/0xdf5a7ac59f174db7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5b5 expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078307:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078310:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078316:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f174dd3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5d8 expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078325:0:6405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f174dd3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5d8 expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078333:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078335:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078341:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000684da15/0xdf5a7ac59f174de8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5e6 expref: 25 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078349:0:6405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000684da15/0xdf5a7ac59f174de8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5e6 expref: 25 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078365:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078368:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078374:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000258fc572/0xdf5a7ac59f174def lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ed expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078383:0:6405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000258fc572/0xdf5a7ac59f174def lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ed expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078391:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078393:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078399:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000c63f95b/0xdf5a7ac59f174df6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5f4 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078409:0:6405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000c63f95b/0xdf5a7ac59f174df6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5f4 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078416:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078419:0:6405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762790.078424:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008bd9074b/0xdf5a7ac59f174e0b lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c609 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078432:0:6405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008bd9074b/0xdf5a7ac59f174e0b lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c609 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.078460:0:6405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008bd9074b/0xdf5a7ac59f174e0b lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c609 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762790.078479:0:6405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fdfe0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074648448:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762790.078557:0:6405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000c63f95b/0xdf5a7ac59f174df6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5f4 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762790.078571:0:6405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2759180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074648576:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762790.078619:0:6405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000258fc572/0xdf5a7ac59f174def lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ed expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762790.078632:0:6405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2758a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074648704:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762790.078674:0:6405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000684da15/0xdf5a7ac59f174de8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5e6 expref: 25 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762790.078686:0:6405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d275b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074648832:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762790.078715:0:6405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f174dd3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5d8 expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762790.078727:0:6405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2758380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074648960:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762790.078761:0:6405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005a11c428/0xdf5a7ac59f174db7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5b5 expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762790.078775:0:6405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d275aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074649088:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762790.078834:0:6405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000021769b91/0xdf5a7ac59f174db0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ae expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762790.078847:0:6405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2758e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074649216:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762790.078880:0:6405:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7b7ed80 going to sleep for 11 seconds 00000100:00100000:1.0:1789762790.079508:0:6405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fdfe0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074648448:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762790.079518:0:6405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2759180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074648576:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762790.080154:0:6405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2758a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074648704:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762790.080169:0:6405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d275b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074648832:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762790.080182:0:6405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d275aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074649088:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762790.080971:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263378043216 time 263328043216 state 0x1 ref 8 00000100:00100000:0.0:1789762790.080988:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ca314000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:2398:x1876702067313792:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762790.085293:0:6405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2758380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074648960:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762790.085302:0:6405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2758e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6405:1876702074649216:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762790.085344:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762790.085380:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cce680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2397:x1876702067317504:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 7215us (7349us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.085387:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 65 00000100:00100000:1.0:1789762790.086011:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ca314000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:2398:x1876702067313792:12345-192.168.203.37@tcp:4:dd.500 Request processed in 5025us (163141us total) trans 4294967560 rc 0/0 00000100:00100000:1.0:1789762790.086019:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 8 00002000:00100000:1.0:1789762790.086029:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x6:0x0] (4194304->5242879) req@ffff9c37ca314000 x1876702067313792/t4294967560(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762790.086039:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ca314000 with x1876702067313792 ext(4194304->5242879) 00002000:00010000:1.0:1789762790.086049:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ca314000 00000100:00100000:2.0:1789762790.112388:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.112451:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067317632 00000100:00100000:1.0:1789762790.112460:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067317632 00002000:00100000:1.0:1789762790.112499:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x7:0x0] (0->1048575) req@ffff9c37d8ccc000 x1876702067317632/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762790.112507:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccc000 with x1876702067317632 ext(0->1048575) 00010000:00010000:1.0:1789762790.112514:0:13300:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 23 pid: 9038 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.112529:0:13300:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 23 pid: 9038 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762790.112535:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9c37d8ccc000 00000100:00100000:1.0:1789762790.112550:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 263318188200 time 263318119682 state 0x0 ref 1 00000100:00100000:1.0:1789762790.112560:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 263318219682 time 263318119682 state 0x1 ref 1 00000100:00100000:1.0:1789762790.112590:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 263318219682 time 263359665458 state 0x0 ref 1 00000100:00100000:1.0:1789762790.112603:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:2395:x1876702067317632:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762790.117244:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2395:x1876702067317632:12345-192.168.203.37@tcp:4:dd.0 Request processed in 4640us (4859us total) trans 4294967561 rc 0/0 00000100:00100000:1.0:1789762790.117255:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1 00000100:00100000:1.0:1789762790.117264:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 263318219682 time 263359665458 state 0x2 ref 0 00002000:00100000:1.0:1789762790.117288:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x7:0x0] (0->1048575) req@ffff9c37d8ccc000 x1876702067317632/t4294967561(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762790.117299:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccc000 with x1876702067317632 ext(0->1048575) 00010000:00010000:1.0:1789762790.117306:0:13300:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 25 pid: 9038 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.117320:0:13300:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 25 pid: 9038 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762790.117329:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9c37d8ccc000 00000100:00100000:2.0:1789762790.118911:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.118974:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067317760 00000100:00100000:0.0:1789762790.118981:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067317760 00010000:00010000:0.0:1789762790.119007:0:9878:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 25 pid: 9038 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.119042:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 66 00000100:00100000:0.0:1789762790.119052:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c93450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2396:x1876702067317760:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762790.119073:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1789762790.119086:0:9878:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x7:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:0.0:1789762790.119090:0:9878:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x7:0x0] updating lvb mtime from disk: 0 -> 1789762789 00002000:00010000:0.0:1789762790.119093:0:9878:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x7:0x0] updating lvb ctime from disk: 0 -> 1789762789 00010000:00010000:0.0:1789762790.119100:0:9878:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 25 pid: 9038 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119114:0:9878:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 25 pid: 9038 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119132:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174dbe lrc: 0/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5bc expref: 25 pid: 9038 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119144:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119147:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119154:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f174dc5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ca expref: 24 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119162:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f174dc5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ca expref: 24 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119169:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119171:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119175:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174dcc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5d1 expref: 24 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119190:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174dcc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5d1 expref: 24 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119195:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119197:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119200:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f174dda lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5c3 expref: 24 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119205:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f174dda lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5c3 expref: 24 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119210:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119212:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119215:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174de1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5df expref: 24 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119220:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174de1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5df expref: 24 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119224:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119225:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119228:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f174e04 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c602 expref: 24 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119233:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f174e04 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c602 expref: 24 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119242:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119244:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119247:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174dfd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5fb expref: 24 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119252:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174dfd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5fb expref: 24 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119256:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119258:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762790.119261:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001cba5766/0xdf5a7ac59f174e12 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c610 expref: 24 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119266:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cba5766/0xdf5a7ac59f174e12 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c610 expref: 24 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.119300:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001cba5766/0xdf5a7ac59f174e12 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c610 expref: 24 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.119311:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9244e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074649344:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762790.119368:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174dfd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5fb expref: 24 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.119376:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9245500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074649472:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762790.119411:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f174e04 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c602 expref: 24 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.119419:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9245880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074649600:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762790.119452:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174de1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5df expref: 24 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.119461:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9245c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074649728:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762790.119494:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f174dda lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5c3 expref: 24 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.119501:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9246d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074649856:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762790.119518:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174dcc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5d1 expref: 24 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.119525:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9244700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074649984:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762790.119539:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f174dc5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ca expref: 24 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.119546:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9247800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074650112:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762790.119560:0:9878:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f61c8780 going to sleep for 11 seconds 00000100:00100000:0.0:1789762790.121731:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9244e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074649344:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762790.121743:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9245500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074649472:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762790.121760:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9245880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074649600:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762790.123232:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9247800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074650112:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762790.123293:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9245c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074649728:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762790.123303:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9246d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074649856:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762790.123315:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9244700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:9878:1876702074649984:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762790.123337:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762790.123364:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c93450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:2396:x1876702067317760:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 4315us (4454us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762790.123370:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 66 00000100:00100000:3.0:1789762790.131037:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263428109960 time 263378109960 state 0x1 ref 7 00000100:00100000:3.0:1789762790.131050:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3b94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:2397:x1876702067313920:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762790.134023:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3b94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2397:x1876702067313920:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2975us (203981us total) trans 4294967562 rc 0/0 00000100:00100000:3.0:1789762790.134030:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 9 00002000:00100000:3.0:1789762790.134036:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x6:0x0] (5242880->6291455) req@ffff9c37e3b94000 x1876702067313920/t4294967562(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762790.134054:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3b94000 with x1876702067313920 ext(5242880->6291455) 00002000:00010000:3.0:1789762790.134058:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e3b94000 00000100:00100000:2.0:1789762790.144887:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.144948:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067317888 00000100:00100000:3.0:1789762790.144955:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067317888 00002000:00100000:3.0:1789762790.144987:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x7:0x0] (3145728->4194303) req@ffff9c37d8ccfb80 x1876702067317888/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762790.144999:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccfb80 with x1876702067317888 ext(3145728->4194303) 00002000:00010000:3.0:1789762790.145004:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ccfb80 00000100:00100000:3.0:1789762790.145021:0:13301:0:(nrs_tbf.c:1585:nrs_tbf_cli_findadd()) unknown.0 NRS: TBF class insert: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 3 deadline 0 time 263392097401 state 0x0 ref 1 00000100:00100000:3.0:1789762790.145035:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 3 deadline 263392197401 time 263392097401 state 0x1 ref 1 00000100:00100000:0.0:1789762790.145173:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263392197401 time 263392246914 state 0x0 ref 1 00000100:00100000:0.0:1789762790.145186:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2395:x1876702067317888:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762790.147486:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2395:x1876702067317888:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2301us (2600us total) trans 4294967351 rc 0/0 00000100:00100000:1.0:1789762790.147493:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 16 00000100:00100000:1.0:1789762790.147501:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263392197401 time 263392246914 state 0x2 ref 0 00002000:00100000:1.0:1789762790.147509:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x7:0x0] (3145728->4194303) req@ffff9c37d8ccfb80 x1876702067317888/t4294967351(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762790.147516:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccfb80 with x1876702067317888 ext(3145728->4194303) 00002000:00010000:1.0:1789762790.147520:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ccfb80 00000100:00100000:2.0:1789762790.148603:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.148639:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067318016 00000100:00100000:1.0:1789762790.148649:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067318016 00002000:00100000:1.0:1789762790.148667:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0x7:0x0] (1048576->2097151) req@ffff9c37d8ccf480 x1876702067318016/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762790.148673:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccf480 with x1876702067318016 ext(1048576->2097151) 00002000:00010000:1.0:1789762790.148682:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ccf480 00000100:00100000:1.0:1789762790.148690:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263392197401 time 263392246914 state 0x0 ref 1 00000100:00100000:1.0:1789762790.148714:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263392346914 time 263392246914 state 0x1 ref 1 00000100:00100000:1.0:1789762790.148733:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263392346914 time 263395811876 state 0x0 ref 1 00000100:00100000:1.0:1789762790.148743:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2396:x1876702067318016:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762790.150966:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:2396:x1876702067318016:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2223us (2361us total) trans 4294967352 rc 0/0 00000100:00100000:1.0:1789762790.150974:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 17 00000100:00100000:1.0:1789762790.150981:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263392346914 time 263395811876 state 0x2 ref 0 00002000:00100000:1.0:1789762790.150989:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0x7:0x0] (1048576->2097151) req@ffff9c37d8ccf480 x1876702067318016/t4294967352(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762790.150996:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccf480 with x1876702067318016 ext(1048576->2097151) 00002000:00010000:1.0:1789762790.151000:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ccf480 00000100:00100000:2.0:1789762790.167551:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.167601:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067318144 00000100:00100000:3.0:1789762790.167606:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067318144 00002000:00100000:3.0:1789762790.167628:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x7:0x0] (5242880->6291455) req@ffff9c37d8ccd180 x1876702067318144/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762790.167636:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccd180 with x1876702067318144 ext(5242880->6291455) 00002000:00010000:3.0:1789762790.167640:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ccd180 00000100:00100000:3.0:1789762790.167648:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263392346914 time 263395811876 state 0x0 ref 1 00000100:00100000:3.0:1789762790.167673:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263395911876 time 263395811876 state 0x1 ref 1 00000100:00100000:3.0:1789762790.167690:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263395911876 time 263414768302 state 0x0 ref 1 00000100:00100000:3.0:1789762790.167709:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:2398:x1876702067318144:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762790.173867:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2398:x1876702067318144:12345-192.168.203.37@tcp:4:dd.0 Request processed in 6156us (6316us total) trans 4294967353 rc 0/0 00000100:00100000:3.0:1789762790.173877:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 18 00000100:00100000:3.0:1789762790.173887:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263395911876 time 263414768302 state 0x2 ref 0 00002000:00100000:3.0:1789762790.173899:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x7:0x0] (5242880->6291455) req@ffff9c37d8ccd180 x1876702067318144/t4294967353(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762790.173910:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccd180 with x1876702067318144 ext(5242880->6291455) 00002000:00010000:3.0:1789762790.173914:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ccd180 00000100:00100000:1.0:1789762790.181096:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263478169734 time 263428169734 state 0x1 ref 6 00000100:00100000:1.0:1789762790.181110:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:2397:x1876702067314048:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762790.183708:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2397:x1876702067314048:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2598us (245291us total) trans 4294967563 rc 0/0 00000100:00100000:1.0:1789762790.183718:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 10 00002000:00100000:1.0:1789762790.183737:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x6:0x0] (6291456->7340031) req@ffff9c37d184c700 x1876702067314048/t4294967563(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762790.183748:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184c700 with x1876702067314048 ext(6291456->7340031) 00002000:00010000:1.0:1789762790.183753:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184c700 00000100:00100000:2.0:1789762790.196867:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.196911:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067318272 00000100:00100000:0.0:1789762790.196948:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067318272 00000100:00100000:2.0:1789762790.196961:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00002000:00100000:0.0:1789762790.196982:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0x7:0x0] (2097152->3145727) req@ffff9c37d8ccca80 x1876702067318272/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762790.196995:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccca80 with x1876702067318272 ext(2097152->3145727) 00002000:00010000:0.0:1789762790.197000:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8ccca80 00000100:00100000:1.0:1789762790.197002:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067318400 00000100:00100000:1.0:1789762790.197010:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067318400 00000100:00100000:0.0:1789762790.197017:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263395911876 time 263414768302 state 0x0 ref 1 00002000:00100000:1.0:1789762790.197034:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0x7:0x0] (6291456->7340031) req@ffff9c37d8ccf100 x1876702067318400/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1789762790.197054:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263414868302 time 263414768302 state 0x1 ref 1 00002000:00100000:1.0:1789762790.197059:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccf100 with x1876702067318400 ext(6291456->7340031) 00002000:00010000:1.0:1789762790.197064:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ccf100 00000100:00100000:1.0:1789762790.197080:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263414868302 time 263414768302 state 0x1 ref 2 00000100:00100000:2.0:1789762790.197101:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263444271606 time 263444171606 state 0x1 ref 2 00000100:00100000:0.0:1789762790.197116:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 1 deadline 263444271606 time 263444192801 state 0x0 ref 2 00000100:00100000:2.0:1789762790.197122:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2395:x1876702067318272:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762790.197134:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2396:x1876702067318400:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762790.200286:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.200344:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067318528 00000100:00100000:1.0:1789762790.200352:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067318528 00002000:00100000:1.0:1789762790.200376:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x7:0x0] (7340032->8388607) req@ffff9c37d9244700 x1876702067318528/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762790.200383:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d9244700 with x1876702067318528 ext(7340032->8388607) 00002000:00010000:1.0:1789762790.200387:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d9244700 00000100:00100000:1.0:1789762790.200417:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 1 deadline 263444292801 time 263444192801 state 0x1 ref 3 00000100:00100000:1.0:1789762790.200439:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263444292801 time 263447517714 state 0x0 ref 3 00000100:00100000:1.0:1789762790.200450:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2395:x1876702067318528:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762790.200869:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:2395:x1876702067318272:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3751us (4002us total) trans 4294967564 rc 0/0 00000100:00100000:3.0:1789762790.200879:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 19 00002000:00100000:3.0:1789762790.200889:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x7:0x0] (2097152->3145727) req@ffff9c37d8ccca80 x1876702067318272/t4294967564(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762790.200901:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccca80 with x1876702067318272 ext(2097152->3145727) 00000100:00100000:0.0:1789762790.200902:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:2396:x1876702067318400:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3772us (3984us total) trans 4294967354 rc 0/0 00002000:00010000:3.0:1789762790.200910:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8ccca80 00000100:00100000:0.0:1789762790.200914:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 20 00002000:00100000:0.0:1789762790.200955:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0x7:0x0] (6291456->7340031) req@ffff9c37d8ccf100 x1876702067318400/t4294967354(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762790.200965:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccf100 with x1876702067318400 ext(6291456->7340031) 00002000:00010000:0.0:1789762790.200971:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ccf100 00000100:00100000:2.0:1789762790.203095:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.203145:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067318656 00000100:00100000:3.0:1789762790.203149:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067318656 00002000:00100000:3.0:1789762790.203168:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0x7:0x0] (5242880->6291455) req@ffff9c37d8ccc380 x1876702067318656/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762790.203175:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccc380 with x1876702067318656 ext(5242880->6291455) 00002000:00010000:3.0:1789762790.203179:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8ccc380 00000100:00100000:3.0:1789762790.203265:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263447617714 time 263447517714 state 0x1 ref 2 00000100:00100000:1.0:1789762790.203281:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263447617714 time 263450353570 state 0x0 ref 2 00000100:00100000:1.0:1789762790.203305:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:2396:x1876702067318656:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762790.203500:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2395:x1876702067318528:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3052us (3214us total) trans 4294967565 rc 0/0 00000100:00100000:2.0:1789762790.203508:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 21 00002000:00100000:2.0:1789762790.203514:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x7:0x0] (7340032->8388607) req@ffff9c37d9244700 x1876702067318528/t4294967565(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762790.203522:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d9244700 with x1876702067318528 ext(7340032->8388607) 00002000:00010000:2.0:1789762790.203526:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d9244700 00000100:00100000:2.0:1789762790.205523:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2396:x1876702067318656:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2222us (2428us total) trans 4294967566 rc 0/0 00000100:00100000:2.0:1789762790.205531:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 22 00000100:00100000:2.0:1789762790.205538:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263447617714 time 263450353570 state 0x2 ref 0 00002000:00100000:2.0:1789762790.205546:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0x7:0x0] (5242880->6291455) req@ffff9c37d8ccc380 x1876702067318656/t4294967566(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762790.205564:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccc380 with x1876702067318656 ext(5242880->6291455) 00002000:00010000:2.0:1789762790.205568:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8ccc380 00000100:00100000:1.0:1789762790.206009:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.206044:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067318784 00000100:00100000:0.0:1789762790.206050:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067318784 00002000:00100000:0.0:1789762790.206069:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x7:0x0] (1048576->2097151) req@ffff9c37d2758e00 x1876702067318784/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762790.206077:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d2758e00 with x1876702067318784 ext(1048576->2097151) 00002000:00010000:0.0:1789762790.206080:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d2758e00 00000100:00100000:0.0:1789762790.206088:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263447617714 time 263450353570 state 0x0 ref 1 00000100:00100000:0.0:1789762790.206113:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263450453570 time 263450353570 state 0x1 ref 1 00000100:00100000:0.0:1789762790.206133:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263450453570 time 263453211268 state 0x0 ref 1 00000100:00100000:0.0:1789762790.206153:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2397:x1876702067318784:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762790.207914:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.207972:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067318912 00000100:00100000:0.0:1789762790.207977:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067318912 00002000:00100000:0.0:1789762790.207994:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0x7:0x0] (7340032->8388607) req@ffff9c37d2758380 x1876702067318912/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762790.208001:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d2758380 with x1876702067318912 ext(7340032->8388607) 00002000:00010000:0.0:1789762790.208004:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d2758380 00000100:00100000:0.0:1789762790.208033:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263453311268 time 263453211268 state 0x1 ref 2 00000100:00100000:0.0:1789762790.208052:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263453311268 time 263455131076 state 0x0 ref 2 00000100:00100000:0.0:1789762790.208061:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:2398:x1876702067318912:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762790.208392:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2397:x1876702067318784:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2240us (2382us total) trans 4294967567 rc 0/0 00000100:00100000:1.0:1789762790.208398:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 23 00002000:00100000:1.0:1789762790.208405:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x7:0x0] (1048576->2097151) req@ffff9c37d2758e00 x1876702067318784/t4294967567(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762790.208412:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d2758e00 with x1876702067318784 ext(1048576->2097151) 00002000:00010000:1.0:1789762790.208415:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d2758e00 00000100:00100000:2.0:1789762790.209646:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.209692:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067319040 00000100:00100000:1.0:1789762790.209698:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067319040 00002000:00100000:1.0:1789762790.209718:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x7:0x0] (0->1048575) req@ffff9c37d8ccea00 x1876702067319040/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762790.209727:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccea00 with x1876702067319040 ext(0->1048575) 00002000:00010000:1.0:1789762790.209731:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ccea00 00000100:00100000:1.0:1789762790.209758:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263455231076 time 263455131076 state 0x1 ref 2 00000100:00100000:2.0:1789762790.209789:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263455231076 time 263456862955 state 0x0 ref 2 00000100:00100000:2.0:1789762790.209803:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2395:x1876702067319040:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762790.212210:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2395:x1876702067319040:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2410us (2565us total) trans 4294967355 rc 0/0 00000100:00100000:2.0:1789762790.212217:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 25 00002000:00100000:2.0:1789762790.212224:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0x7:0x0] (0->1048575) req@ffff9c37d8ccea00 x1876702067319040/t4294967355(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762790.212231:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccea00 with x1876702067319040 ext(0->1048575) 00002000:00010000:2.0:1789762790.212234:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ccea00 00000100:00100000:0.0:1789762790.213787:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2398:x1876702067318912:12345-192.168.203.37@tcp:4:dd.0 Request processed in 5725us (5871us total) trans 4294967356 rc 0/0 00000100:00100000:0.0:1789762790.213799:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 24 00000100:00100000:0.0:1789762790.213808:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263455231076 time 263456862955 state 0x2 ref 0 00002000:00100000:0.0:1789762790.213820:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0x7:0x0] (7340032->8388607) req@ffff9c37d2758380 x1876702067318912/t4294967356(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762790.213829:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d2758380 with x1876702067318912 ext(7340032->8388607) 00002000:00010000:0.0:1789762790.213833:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d2758380 00000100:00100000:1.0:1789762790.213896:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.213962:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067319168 00000100:00100000:0.0:1789762790.213974:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067319168 00002000:00100000:0.0:1789762790.213998:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x7:0x0] (4194304->5242879) req@ffff9c37d275aa00 x1876702067319168/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762790.214007:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d275aa00 with x1876702067319168 ext(4194304->5242879) 00002000:00010000:0.0:1789762790.214011:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d275aa00 00000100:00100000:0.0:1789762790.214022:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263455231076 time 263456862955 state 0x0 ref 1 00000100:00100000:0.0:1789762790.214056:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263456962955 time 263456862955 state 0x1 ref 1 00000100:00100000:2.0:1789762790.214085:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263456962955 time 263461159275 state 0x0 ref 1 00000100:00100000:2.0:1789762790.214100:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2397:x1876702067319168:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762790.216468:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2397:x1876702067319168:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2370us (2570us total) trans 4294967568 rc 0/0 00000100:00100000:2.0:1789762790.216476:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 26 00000100:00100000:2.0:1789762790.216482:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263456962955 time 263461159275 state 0x2 ref 0 00002000:00100000:2.0:1789762790.216491:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x7:0x0] (4194304->5242879) req@ffff9c37d275aa00 x1876702067319168/t4294967568(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762790.216504:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d275aa00 with x1876702067319168 ext(4194304->5242879) 00002000:00010000:2.0:1789762790.216508:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d275aa00 00000100:00100000:2.0:1789762790.216575:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.216614:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067319296 00000100:00100000:1.0:1789762790.216619:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067319296 00002000:00100000:1.0:1789762790.216637:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0x7:0x0] (3145728->4194303) req@ffff9c37d8ccce00 x1876702067319296/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762790.216644:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccce00 with x1876702067319296 ext(3145728->4194303) 00002000:00010000:1.0:1789762790.216652:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8ccce00 00000100:00100000:1.0:1789762790.216660:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263456962955 time 263461159275 state 0x0 ref 1 00000100:00100000:1.0:1789762790.216688:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263461259275 time 263461159275 state 0x1 ref 1 00000100:00100000:3.0:1789762790.216718:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263461259275 time 263463791604 state 0x0 ref 1 00000100:00100000:3.0:1789762790.216732:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2396:x1876702067319296:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762790.218764:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.218797:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067319424 00000100:00100000:0.0:1789762790.218802:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067319424 00002000:00100000:0.0:1789762790.218820:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0x7:0x0] (4194304->5242879) req@ffff9c37d8ccd500 x1876702067319424/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762790.218827:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccd500 with x1876702067319424 ext(4194304->5242879) 00002000:00010000:0.0:1789762790.218830:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ccd500 00000100:00100000:0.0:1789762790.218856:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263463891604 time 263463791604 state 0x1 ref 2 00000100:00100000:0.0:1789762790.218873:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263463891604 time 263465951895 state 0x0 ref 2 00000100:00100000:0.0:1789762790.218884:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2398:x1876702067319424:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762790.219092:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2396:x1876702067319296:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2362us (2514us total) trans 4294967569 rc 0/0 00000100:00100000:2.0:1789762790.219098:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 27 00002000:00100000:2.0:1789762790.219106:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x7:0x0] (3145728->4194303) req@ffff9c37d8ccce00 x1876702067319296/t4294967569(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762790.219113:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccce00 with x1876702067319296 ext(3145728->4194303) 00002000:00010000:2.0:1789762790.219117:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8ccce00 00000100:00100000:0.0:1789762790.219468:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.219543:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067319680 00000100:00100000:2.0:1789762790.219549:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067319680 00000100:00100000:2.0:1789762790.219581:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 43 00000100:00100000:2.0:1789762790.219587:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:13327:x1876702067319680:12345-192.168.203.37@tcp:35:dd.0 00000001:00080000:2.0:1789762790.219793:0:6421:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967367 00000100:00100000:2.0:1789762790.219856:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:13327:x1876702067319680:12345-192.168.203.37@tcp:35:dd.0 Request processed in 268us (387us total) trans 4294967368 rc 0/0 00000100:00100000:2.0:1789762790.219866:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 43 00000020:00080000:2.0:1789762790.219871:0:6421:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1789762790.221499:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.221531:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067319808 00000100:00100000:3.0:1789762790.221537:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067319808 00002000:00100000:3.0:1789762790.221558:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x7:0x0] (6291456->7340031) req@ffff9c37d275b100 x1876702067319808/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762790.221569:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d275b100 with x1876702067319808 ext(6291456->7340031) 00002000:00010000:3.0:1789762790.221574:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d275b100 00000100:00100000:3.0:1789762790.221610:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263466051895 time 263465951895 state 0x1 ref 2 00000100:00100000:2.0:1789762790.221641:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2398:x1876702067319424:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2756us (2874us total) trans 4294967357 rc 0/0 00000100:00100000:3.0:1789762790.221642:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263466051895 time 263468716476 state 0x0 ref 2 00000100:00100000:3.0:1789762790.221661:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2397:x1876702067319808:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762790.221665:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 28 00002000:00100000:2.0:1789762790.221678:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0x7:0x0] (4194304->5242879) req@ffff9c37d8ccd500 x1876702067319424/t4294967357(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762790.221697:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccd500 with x1876702067319424 ext(4194304->5242879) 00002000:00010000:2.0:1789762790.221702:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ccd500 00000100:00100000:2.0:1789762790.224391:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2397:x1876702067319808:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2732us (2894us total) trans 4294967570 rc 0/0 00000100:00100000:2.0:1789762790.224399:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 29 00000100:00100000:2.0:1789762790.224406:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263466051895 time 263468716476 state 0x2 ref 0 00002000:00100000:2.0:1789762790.224414:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x7:0x0] (6291456->7340031) req@ffff9c37d275b100 x1876702067319808/t4294967570(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/448 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762790.224421:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d275b100 with x1876702067319808 ext(6291456->7340031) 00002000:00010000:2.0:1789762790.224424:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d275b100 00000100:00100000:2.0:1789762790.231169:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263528242520 time 263478242520 state 0x1 ref 5 00000100:00100000:2.0:1789762790.231184:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2397:x1876702067314176:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762790.233709:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2397:x1876702067314176:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2526us (288404us total) trans 4294967571 rc 0/0 00000100:00100000:0.0:1789762790.233717:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 11 00002000:00100000:0.0:1789762790.233732:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x6:0x0] (7340032->8388607) req@ffff9c37d8c8d880 x1876702067314176/t4294967571(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762790.233739:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c8d880 with x1876702067314176 ext(7340032->8388607) 00002000:00010000:0.0:1789762790.233743:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8c8d880 00000100:00100000:1.0:1789762790.281267:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263578328643 time 263528328643 state 0x1 ref 4 00000100:00100000:1.0:1789762790.281296:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2397:x1876702067314304:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762790.284121:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2397:x1876702067314304:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2835us (332110us total) trans 4294967358 rc 0/0 00000100:00100000:2.0:1789762790.284130:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 12 00002000:00100000:2.0:1789762790.284139:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0x6:0x0] (4194304->5242879) req@ffff9c37d184ce00 x1876702067314304/t4294967358(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762790.284147:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184ce00 with x1876702067314304 ext(4194304->5242879) 00002000:00010000:2.0:1789762790.284152:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184ce00 00000100:00100000:2.0:1789762790.331345:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263628410447 time 263578410447 state 0x1 ref 3 00000100:00100000:2.0:1789762790.331384:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2397:x1876702067314432:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762790.337775:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2397:x1876702067314432:12345-192.168.203.37@tcp:4:dd.500 Request processed in 6398us (379012us total) trans 4294967359 rc 0/0 00000100:00100000:2.0:1789762790.337783:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 13 00002000:00100000:2.0:1789762790.337793:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0x6:0x0] (5242880->6291455) req@ffff9c37d184ea00 x1876702067314432/t4294967359(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762790.337801:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184ea00 with x1876702067314432 ext(5242880->6291455) 00002000:00010000:2.0:1789762790.337808:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184ea00 00000100:00100000:0.0:1789762790.381432:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263678497219 time 263628497219 state 0x1 ref 2 00000100:00100000:0.0:1789762790.381456:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bc7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2397:x1876702067314560:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762790.384365:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bc7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2397:x1876702067314560:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2917us (398327us total) trans 4294967360 rc 0/0 00000100:00100000:2.0:1789762790.384374:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 14 00002000:00100000:2.0:1789762790.384383:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0x6:0x0] (6291456->7340031) req@ffff9c37e3bc7100 x1876702067314560/t4294967360(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762790.384393:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bc7100 with x1876702067314560 ext(6291456->7340031) 00002000:00010000:2.0:1789762790.384400:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3bc7100 00000100:00100000:1.0:1789762790.431536:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263678497219 time 263678599199 state 0x0 ref 1 00000100:00100000:1.0:1789762790.431564:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2397:x1876702067314688:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762790.434756:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:2397:x1876702067314688:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3202us (438005us total) trans 4294967361 rc 0/0 00000100:00100000:2.0:1789762790.434764:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 15 00000100:00100000:2.0:1789762790.434772:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263678497219 time 263678599199 state 0x2 ref 0 00002000:00100000:2.0:1789762790.434782:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x6:0x0] (7340032->8388607) req@ffff9c37fd9c3800 x1876702067314688/t4294967361(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:570/0 lens 488/448 e 0 to 0 dl 1789762800 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762790.434804:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fd9c3800 with x1876702067314688 ext(7340032->8388607) 00002000:00010000:2.0:1789762790.434808:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fd9c3800 00000100:00100000:0.0:1789762790.469709:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.469832:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067319936 00000100:00100000:2.0:1789762790.469845:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067319936 00000100:00100000:0.0:1789762790.469950:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 24 00000100:00100000:0.0:1789762790.469969:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:6823:x1876702067319936:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762790.469989:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 8 locks, starting at 0 00000100:00100000:1.0:1789762790.470004:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00002000:00010000:0.0:1789762790.470013:0:7218:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0x6:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:0.0:1789762790.470017:0:7218:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0x6:0x0] updating lvb mtime from disk: 0 -> 1789762789 00002000:00010000:0.0:1789762790.470020:0:7218:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0x6:0x0] updating lvb ctime from disk: 0 -> 1789762789 00002000:00010000:0.0:1789762790.470022:0:7218:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000400:0x6:0x0] updating lvb blocks from disk: 1 -> 2051 00000100:00100000:2.0:1789762790.470049:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067320064 00010000:00010000:0.0:1789762790.470054:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174d7f lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c599 expref: 43 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762790.470055:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067320064 00002000:00010000:0.0:1789762790.470094:0:7218:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0x7:0x0] updating lvb size from disk: 3145728 -> 8388608 00000100:00100000:2.0:1789762790.470095:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 25 00002000:00010000:0.0:1789762790.470103:0:7218:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0x7:0x0] updating lvb mtime from disk: 1789762789 -> 1789762790 00002000:00010000:0.0:1789762790.470107:0:7218:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0x7:0x0] updating lvb ctime from disk: 1789762789 -> 1789762790 00000100:00100000:2.0:1789762790.470115:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:6358:x1876702067320064:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762790.470125:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008bd9074b/0xdf5a7ac59f174e0b lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c609 expref: 42 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.470129:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 8 locks, starting at 0 00002000:00010000:2.0:1789762790.470153:0:6404:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x6:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:2.0:1789762790.470157:0:6404:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x6:0x0] updating lvb mtime from disk: 0 -> 1789762789 00002000:00010000:2.0:1789762790.470163:0:6404:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x6:0x0] updating lvb ctime from disk: 0 -> 1789762789 00002000:00010000:2.0:1789762790.470175:0:6404:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x6:0x0] updating lvb blocks from disk: 1 -> 4103 00010000:00010000:2.0:1789762790.470207:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000682c79ca/0xdf5a7ac59f174d71 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c592 expref: 41 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.470211:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021769b91/0xdf5a7ac59f174db0 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 8 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ae expref: 41 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.470266:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000c63f95b/0xdf5a7ac59f174df6 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 7 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5f4 expref: 40 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762790.470279:0:6404:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x7:0x0] updating lvb size from disk: 1048576 -> 8388608 00002000:00010000:2.0:1789762790.470283:0:6404:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x7:0x0] updating lvb mtime from disk: 1789762789 -> 1789762790 00002000:00010000:2.0:1789762790.470286:0:6404:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x7:0x0] updating lvb ctime from disk: 1789762789 -> 1789762790 00010000:00010000:0.0:1789762790.470288:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000684da15/0xdf5a7ac59f174de8 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5e6 expref: 39 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.470311:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f174dfd lrc: 0/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5fb expref: 40 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.470312:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f174dd3 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 5 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5d8 expref: 38 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.470349:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005a11c428/0xdf5a7ac59f174db7 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 4 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5b5 expref: 37 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.470383:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000258fc572/0xdf5a7ac59f174def lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ed expref: 36 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.470389:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f174dc5 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 8 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5ca expref: 39 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.470397:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762790.470443:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f174dcc lrc: 0/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 7 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5d1 expref: 38 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.470458:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:6823:x1876702067319936:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 496us (759us total) trans 0 rc 0/0 00010000:00010000:2.0:1789762790.470462:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f174dda lrc: 0/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 6 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5c3 expref: 37 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.470484:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 24 00010000:00010000:2.0:1789762790.470492:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f174e04 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 5 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c602 expref: 36 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.470508:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cba5766/0xdf5a7ac59f174e12 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 4 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c610 expref: 35 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.470521:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174de1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5df expref: 34 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.470531:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762790.470568:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:6358:x1876702067320064:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 460us (571us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762790.470574:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 25 00000100:00100000:0.0:1789762790.506067:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.506156:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067320320 00000100:00100000:2.0:1789762790.506170:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067320320 00000100:00100000:2.0:1789762790.506230:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 178 00000100:00100000:2.0:1789762790.506241:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9247800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:13349:x1876702067320320:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762790.506263:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762790.506293:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e19 lrc: 2/0,0 mode: --/CR res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c617 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.506377:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e20 lrc: 3/1,0 mode: --/PR res: [0x240000402:0xc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.506389:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e20 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.506399:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e20 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762790.506406:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xc:0x0]: returning size 16777216 00010000:00010000:2.0:1789762790.506509:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e19 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c617 expref: 26 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.506526:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e20 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c617 expref: 25 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.506532:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xc:0x0].0x0 (ffff9c37d3beef00) 00010000:00010000:2.0:1789762790.506535:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762790.506537:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b10400, rc 0) 00000100:00100000:2.0:1789762790.506596:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9247800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:13349:x1876702067320320:12345-192.168.203.37@tcp:101:dd.0 Request processed in 358us (542us total) trans 4294967369 rc 0/0 00000100:00100000:2.0:1789762790.506601:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 178 00000100:00100000:1.0:1789762790.508275:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.508320:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.508329:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067320448 00000100:00100000:0.0:1789762790.508351:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067320448 00000100:00100000:1.0:1789762790.508363:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.508398:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067320576 00000100:00100000:2.0:1789762790.508406:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067320576 00000100:00100000:3.0:1789762790.508426:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067320704 00000100:00100000:0.0:1789762790.508436:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 26 00000100:00100000:3.0:1789762790.508438:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067320704 00000100:00100000:0.0:1789762790.508451:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2396:x1876702067320448:12345-192.168.203.37@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1789762790.508464:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 123 00010000:00010000:0.0:1789762790.508469:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1789762790.508476:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:13351:x1876702067320576:12345-192.168.203.37@tcp:101:dd.0 00000100:00100000:1.0:1789762790.508483:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 124 00010000:00010000:2.0:1789762790.508499:0:8052:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762790.508509:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e20 lrc: 0/0,0 mode: --/PR res: [0x240000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c617 expref: 25 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762790.508509:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:13349:x1876702067320704:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762790.508516:0:8052:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f174e2e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c61e expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.508540:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762790.508560:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f174e35 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c625 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762790.508560:0:8052:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x6:0x0] initial LVB size: 8388608, mtime: 0x6aad9ce5, atime: 0x0, ctime: 0x6aad9ce5, blocks: 0x1007 00010000:00010000:0.0:1789762790.508565:0:6406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xc:0x0].0x0 (ffff9c37d3beef00) 00010000:00010000:0.0:1789762790.508584:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762790.508587:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x6:0x0].0x0 since=263s v=(0 0) 00010000:00010000:2.0:1789762790.508594:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762790.508604:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f174e2e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c61e expref: 36 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.508615:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762790.508621:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762790.508636:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f174e35 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c625 expref: 36 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.508643:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:2396:x1876702067320448:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 192us (363us total) trans 0 rc 0/0 00010000:00010000:2.0:1789762790.508659:0:8052:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f174e2e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c61e expref: 36 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.508660:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f174e35 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c625 expref: 36 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.508668:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 26 00010000:00010000:2.0:1789762790.508695:0:8052:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b11c00, rc 0) 00010000:00010000:1.0:1789762790.508696:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a95800, rc 0) 00000100:00100000:2.0:1789762790.508730:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:13351:x1876702067320576:12345-192.168.203.37@tcp:101:dd.0 Request processed in 257us (420us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.508736:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:13349:x1876702067320704:12345-192.168.203.37@tcp:101:dd.0 Request processed in 235us (372us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762790.508743:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 123 00000100:00100000:1.0:1789762790.508751:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 124 00000100:00100000:3.0:1789762790.508876:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.508990:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067320832 00000100:00100000:1.0:1789762790.508998:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067320832 00000100:00100000:1.0:1789762790.509046:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 125 00000100:00100000:1.0:1789762790.509057:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ffcc9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:13350:x1876702067320832:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762790.509065:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762790.509072:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f174e3c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x6:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c62c expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.509084:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762790.509087:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762790.509091:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f174e3c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c62c expref: 36 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.509100:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f174e3c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c62c expref: 36 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.509106:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a95600, rc 0) 00000100:00100000:1.0:1789762790.509141:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ffcc9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:13350:x1876702067320832:12345-192.168.203.37@tcp:101:dd.0 Request processed in 88us (269us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.509147:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 125 00000100:00100000:0.0:1789762790.513459:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.513510:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067320960 00000100:00100000:2.0:1789762790.513516:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067320960 00002000:00100000:2.0:1789762790.513543:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0x6:0x0] (0->1048575) req@ffff9c37d9245880 x1876702067320960/t0(0) o3->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/0 e 0 to 0 dl 1789762801 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762790.513552:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d9245880 with x1876702067320960 ext(0->1048575) 00002000:00010000:2.0:1789762790.513555:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d9245880 00000100:00100000:2.0:1789762790.513573:0:8057:0:(nrs_tbf.c:1585:nrs_tbf_cli_findadd()) unknown.0 NRS: TBF class insert: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 3 deadline 0 time 263760647948 state 0x0 ref 1 00000100:00100000:2.0:1789762790.513581:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 3 deadline 263760747948 time 263760647948 state 0x1 ref 1 00000100:00100000:2.0:1789762790.513612:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263760747948 time 263760689543 state 0x0 ref 1 00000100:00100000:2.0:1789762790.513623:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2396:x1876702067320960:12345-192.168.203.37@tcp:3:dd.0 00000100:00100000:2.0:1789762790.517775:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2396:x1876702067320960:12345-192.168.203.37@tcp:3:dd.0 Request processed in 4153us (4318us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762790.517784:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 30 00000100:00100000:2.0:1789762790.517791:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263760747948 time 263760689543 state 0x2 ref 0 00002000:00100000:2.0:1789762790.517802:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0x6:0x0] (0->1048575) req@ffff9c37d9245880 x1876702067320960/t0(0) o3->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:571/0 lens 488/440 e 0 to 0 dl 1789762801 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762790.517809:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d9245880 with x1876702067320960 ext(0->1048575) 00002000:00010000:2.0:1789762790.517813:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d9245880 00000100:00100000:2.0:1789762790.552268:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.552342:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067321216 00000100:00100000:3.0:1789762790.552356:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067321216 00000100:00100000:3.0:1789762790.552422:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 44 00000100:00100000:3.0:1789762790.552436:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:13359:x1876702067321216:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:3.0:1789762790.552728:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:13359:x1876702067321216:12345-192.168.203.37@tcp:35:dd.0 Request processed in 294us (465us total) trans 4294967370 rc 0/0 00000100:00100000:3.0:1789762790.552755:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 44 00000100:00100000:3.0:1789762790.563828:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.563880:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067321344 00000100:00100000:0.0:1789762790.563888:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067321344 00000100:00100000:0.0:1789762790.563967:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 179 00000100:00100000:0.0:1789762790.563978:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:13365:x1876702067321344:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:0.0:1789762790.563996:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762790.564025:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174e43 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c633 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.564092:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174e4a lrc: 3/1,0 mode: --/PR res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.564111:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174e4a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.564125:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174e4a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762790.564180:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xb:0x0]: returning size 16384 00000004:00200000:0.0:1789762790.564185:0:6420:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762790.564202:0:6420:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174e4a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.564229:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174e43 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c633 expref: 27 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.564266:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174e4a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c633 expref: 26 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.564277:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:0.0:1789762790.564281:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762790.564285:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa7200, rc 0) 00000100:00100000:0.0:1789762790.564329:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:13365:x1876702067321344:12345-192.168.203.37@tcp:101:rm.0 Request processed in 353us (503us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762790.564336:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 179 00000100:00100000:0.0:1789762790.565396:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.565440:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067321600 00000100:00100000:3.0:1789762790.565445:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067321600 00000100:00100000:3.0:1789762790.565474:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 180 00000100:00100000:3.0:1789762790.565480:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:13365:x1876702067321600:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:3.0:1789762790.565489:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762790.565499:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f174e51 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c63a expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762790.565548:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xb:0x0]: returning size 16384 00010000:00010000:3.0:1789762790.565621:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f174e51 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c63a expref: 28 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.565632:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37c72ba300) 00010000:00010000:3.0:1789762790.565635:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762790.565638:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f174e51 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c63a expref: 28 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.565650:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff935e00, rc 0) 00000100:00100000:3.0:1789762790.565681:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:13365:x1876702067321600:12345-192.168.203.37@tcp:101:rm.0 Request processed in 201us (287us total) trans 4294967371 rc 301/301 00000100:00100000:3.0:1789762790.565686:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 180 00000100:00100000:2.0:1789762790.567369:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.567424:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067321728 00000100:00100000:0.0:1789762790.567434:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067321728 00000100:00100000:0.0:1789762790.567475:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 45 00000100:00100000:0.0:1789762790.567483:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:13365:x1876702067321728:12345-192.168.203.37@tcp:37:rm.0 00000100:00100000:2.0:1789762790.569358:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:13365:x1876702067321728:12345-192.168.203.37@tcp:37:rm.0 Request processed in 1873us (1990us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762790.569365:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 45 00000100:00100000:1.0:1789762790.570683:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.570723:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067321984 00000100:00100000:3.0:1789762790.570731:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067321984 00000100:00100000:3.0:1789762790.570771:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 46 00000100:00100000:3.0:1789762790.570784:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbd450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:13365:x1876702067321984:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:3.0:1789762790.570907:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbd450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:13365:x1876702067321984:12345-192.168.203.37@tcp:35:rm.0 Request processed in 129us (224us total) trans 4294967372 rc 0/0 00000100:00100000:3.0:1789762790.570956:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 46 00000100:00100000:1.0:1789762790.571848:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.571885:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067322240 00000100:00100000:3.0:1789762790.571889:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067322240 00000100:00100000:3.0:1789762790.571941:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 181 00000100:00100000:3.0:1789762790.571948:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:13365:x1876702067322240:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:3.0:1789762790.571957:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762790.571968:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174e5f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c641 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.572009:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174e66 lrc: 3/1,0 mode: --/CR res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.572018:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174e66 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0xb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.572025:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174e66 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0xb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762790.572031:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xb:0x0]: returning size 16384 00010000:00010000:3.0:1789762790.572072:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174e5f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c641 expref: 31 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.572085:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174e66 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0xb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c641 expref: 30 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.572091:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:3.0:1789762790.572099:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1789762790.572101:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff935c00, rc 0) 00000100:00100000:3.0:1789762790.572128:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:13365:x1876702067322240:12345-192.168.203.37@tcp:101:rm.0 Request processed in 181us (282us total) trans 4294967373 rc 0/0 00000100:00100000:3.0:1789762790.572134:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 181 00000100:00100000:0.0:1789762790.573030:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.573108:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067322368 00000100:00100000:2.0:1789762790.573115:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067322368 00000100:00100000:2.0:1789762790.573160:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 182 00000100:00080000:1.0:1789762790.573172:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762790.573173:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13365:x1876702067322368:12345-192.168.203.37@tcp:41:rm.0 00000004:00000024:2.0:1789762790.573216:0:13389:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7719616512 avail 7715422208 00000100:00100000:2.0:1789762790.573257:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13365:x1876702067322368:12345-192.168.203.37@tcp:41:rm.0 Request processed in 91us (232us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762790.573269:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 182 00000100:00100000:1.0:1789762790.574140:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.574182:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067322624 00000100:00100000:0.0:1789762790.574186:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067322624 00000100:00100000:0.0:1789762790.574212:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 183 00000100:00100000:0.0:1789762790.574218:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:13365:x1876702067322624:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:0.0:1789762790.574258:0:6418:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1789762790.574285:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174d9b lrc: 0/0,0 mode: --/PR res: [0x240000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c5a0 expref: 30 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.574305:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xd:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762790.574322:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7200/0xdf5a7ac59f174e4a lrc: 0/0,0 mode: --/PR res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c633 expref: 29 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.574333:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:0.0:1789762790.574337:0:6418:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762790.574353:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174e74 lrc: 3/0,1 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.574368:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174e74 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.574378:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174e74 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.574389:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6400/0xdf5a7ac59f174e7b lrc: 3/0,1 mode: --/PW res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.574398:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6400/0xdf5a7ac59f174e7b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.574405:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6400/0xdf5a7ac59f174e7b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.574463:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174e82 lrc: 3/0,1 mode: --/EX res: [0x240000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.574477:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174e82 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.574482:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174e82 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762790.575339:0:6418:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].6, 64 off8576 00000004:00080000:0.0:1789762790.575350:0:6418:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 263822431209ns, next at 264822428470ns, rc = 0 00000040:00080000:0.0:1789762790.575496:0:6418:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].6, 64 off8576 00000004:00080000:0.0:1789762790.575504:0:6418:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 263822585465ns, next at 264822583939ns, rc = 0 00010000:00010000:0.0:1789762790.575557:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174e89 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575565:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174e89 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575570:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174e89 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575575:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174e89 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0xd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575580:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174e89 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0xd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575584:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174e89 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0xd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575602:0:6418:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762790.575614:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174e82 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575633:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174e82 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0xd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575639:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174e82 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575646:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762790.575651:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174e82 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575660:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174e82 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0xd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.575664:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762790.575672:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174e89 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0xd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575673:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xd:0x0].0x0 (ffff9c37e125ae00) 00010000:00010000:0.0:1789762790.575681:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762790.575686:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762790.575688:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174e82 lrc: 2/0,0 mode: --/EX res: [0x240000402:0xd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.575692:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174e89 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0xd:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575704:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174e82 lrc: 0/0,0 mode: --/EX res: [0x240000402:0xd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.575705:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174e89 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0xd:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.575726:0:8069:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xd:0x0].0x0 (ffff9c37e125ae00) 00010000:00010000:2.0:1789762790.575730:0:8069:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762790.575734:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174e74 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.575734:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174e89 lrc: 1/0,0 mode: --/PW res: [0x240000402:0xd:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.575748:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7a00/0xdf5a7ac59f174e89 lrc: 0/0,0 mode: --/PW res: [0x240000402:0xd:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575754:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174e74 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.575763:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762790.575765:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174e74 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.575766:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762790.575769:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762790.575777:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174e74 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575792:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762790.575795:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174e74 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575801:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174e74 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575811:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:0.0:1789762790.575814:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762790.575817:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174e74 lrc: 3/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575823:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174e74 lrc: 0/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762790.575834:0:6418:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009a3479a9 reply state 0000000016414b89 transno 4294967374 ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6400/0xdf5a7ac59f174e7b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575846:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6400/0xdf5a7ac59f174e7b lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.575853:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x2b0e463d (ffff9c37e125aa00) 00010000:00010000:0.0:1789762790.575856:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762790.575858:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762790.575903:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:13365:x1876702067322624:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1682us (1762us total) trans 4294967374 rc 0/0 00000100:00100000:0.0:1789762790.575912:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 183 00000100:00100000:1.0:1789762790.577053:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.577084:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067322880 00000100:00100000:2.0:1789762790.577090:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067322880 00000100:00100000:2.0:1789762790.577119:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 184 00000100:00100000:2.0:1789762790.577126:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:13365:x1876702067322880:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:2.0:1789762790.577147:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762790.577158:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11600/0xdf5a7ac59f174d5c lrc: 0/0,0 mode: --/PR res: [0x240000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c58b expref: 30 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577180:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xc:0x0].0x0 (ffff9c37d3beef00) 00010000:00010000:2.0:1789762790.577186:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762790.577194:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e90 lrc: 3/0,1 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577202:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e90 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577207:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e90 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577213:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e97 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xb:0x0].0x73550ecb bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577218:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e97 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577223:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e97 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577259:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174e9e lrc: 3/0,1 mode: --/EX res: [0x240000402:0xc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577264:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174e9e lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577268:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174e9e lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789762790.577542:0:6419:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].7, 64 off8640 00000040:00080000:2.0:1789762790.577554:0:6419:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].7, 64 off8640 00010000:00010000:2.0:1789762790.577595:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174ea5 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577602:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174ea5 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577607:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174ea5 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577612:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174ea5 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0xc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577616:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174ea5 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0xc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577620:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174ea5 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0xc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577635:0:6419:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762790.577643:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174e9e lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577648:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174e9e lrc: 2/0,0 mode: EX/EX res: [0x240000402:0xc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.577650:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762790.577653:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174e9e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.577658:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174ea5 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0xc:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577659:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762790.577663:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174e9e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.577673:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762790.577675:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174e9e lrc: 4/0,0 mode: EX/EX res: [0x240000402:0xc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.577678:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174ea5 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0xc:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.577699:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174ea5 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0xc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577702:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xc:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762790.577705:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762790.577708:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174e9e lrc: 2/0,0 mode: --/EX res: [0x240000402:0xc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577714:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11200/0xdf5a7ac59f174e9e lrc: 0/0,0 mode: --/EX res: [0x240000402:0xc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.577716:0:6407:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xc:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:3.0:1789762790.577720:0:6407:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789762790.577723:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174ea5 lrc: 1/0,0 mode: --/PW res: [0x240000402:0xc:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577726:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e90 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.577729:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10e00/0xdf5a7ac59f174ea5 lrc: 0/0,0 mode: --/PW res: [0x240000402:0xc:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577734:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e90 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577742:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e90 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577746:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e90 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.577747:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762790.577750:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762790.577753:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762790.577754:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762790.577757:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e90 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577762:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e90 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577768:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:2.0:1789762790.577769:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762790.577771:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e90 lrc: 3/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577781:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11800/0xdf5a7ac59f174e90 lrc: 0/0,0 mode: --/CW res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762790.577789:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ce6600fa reply state 000000006d490721 transno 4294967375 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e97 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577795:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e97 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.577799:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x73550ecb (ffff9c37d3bee200) 00010000:00010000:2.0:1789762790.577801:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762790.577803:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762790.577835:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:13365:x1876702067322880:12345-192.168.203.37@tcp:36:rm.0 Request processed in 709us (783us total) trans 4294967375 rc 0/0 00000100:00100000:2.0:1789762790.577840:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 184 00000100:00100000:3.0:1789762790.580052:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.580077:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.580084:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067323264 00000100:00100000:2.0:1789762790.580090:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067323264 00000100:00100000:0.0:1789762790.580112:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067323008 00000100:00100000:0.0:1789762790.580118:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067323008 00000100:00100000:2.0:1789762790.580122:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 47 00000100:00100000:2.0:1789762790.580132:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:13365:x1876702067323264:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:0.0:1789762790.580164:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 27 00000100:00100000:0.0:1789762790.580182:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067323008:12345-192.168.203.37@tcp:103:rm.0 00010000:00010000:0.0:1789762790.580194:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762790.580212:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174cbb lrc: 0/0,0 mode: --/PR res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c553 expref: 7 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.580228:0:8070:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125ac00) 00010000:00010000:0.0:1789762790.580236:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762790.580241:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:13365:x1876702067323264:12345-192.168.203.37@tcp:35:rm.0 Request processed in 111us (189us total) trans 4294967376 rc 0/0 00000100:00100000:2.0:1789762790.580249:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 47 00000100:00100000:0.0:1789762790.580271:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067323008:12345-192.168.203.37@tcp:103:rm.0 Request processed in 90us (190us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762790.580281:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 27 00000100:00100000:1.0:1789762790.581067:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.581094:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067323520 00000100:00100000:0.0:1789762790.581099:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067323520 00000100:00100000:0.0:1789762790.581124:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 185 00000100:00100000:0.0:1789762790.581130:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:13365:x1876702067323520:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:0.0:1789762790.581150:0:6420:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1789762790.581160:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ff935c00/0xdf5a7ac59f174e66 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xb:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c641 expref: 32 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.581173:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7fc7400/0xdf5a7ac59f174d40 lrc: 0/0,0 mode: --/PR res: [0x240000402:0xb:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c584 expref: 31 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.581179:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:0.0:1789762790.581181:0:6420:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762790.581207:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.581213:0:6420:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.581226:0:6420:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.581243:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9245f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074650240:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1789762790.581284:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762790.581312:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074650240 00000100:00100000:2.0:1789762790.581318:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074650240 00000100:00100000:0.0:1789762790.581339:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7fbe600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762790.581356:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:2.0:1789762790.581364:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074650240:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1789762790.581373:0:13302:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762790.581389:0:13302:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f174eb3 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f174eac expref: -99 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.581407:0:13302:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174cd0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c561 expref: 5 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.581414:0:13302:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f174eb3 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f174eac expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.581430:0:13302:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174cd0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c561 expref: 5 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.581440:0:13302:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174cd0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c561 expref: 5 pid: 7744 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762790.581448:0:13302:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cced80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:13302:1876702074650368:192.168.203.37@tcp:104: 00000100:00100000:2.0:1789762790.581489:0:13302:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ee88d300 going to sleep for 11 seconds 00000100:00100000:2.0:1789762790.582218:0:13302:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cced80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:13302:1876702074650368:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762790.582237:0:13302:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f174eb3 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174eac expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.582243:0:13302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762790.582245:0:13302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762790.582248:0:13302:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f174eb3 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174eac expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.582253:0:13302:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b11a00, rc 0) 00000100:00100000:2.0:1789762790.582284:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:6420:x1876702074650240:12345-0@lo:101:mdt00_002.0 Request processed in 922us (1000us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762790.582296:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000100:00100000:0.0:1789762790.582314:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9245f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074650240:0@lo:101:mdt00_002.0 00010000:00010000:0.0:1789762790.582325:0:6420:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.582332:0:6420:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762790.582474:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.582507:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067323648 00000100:00100000:0.0:1789762790.582511:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067323648 00010000:00010000:0.0:1789762790.582525:0:7218:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174cd0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c561 expref: 6 pid: 7744 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.582553:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 67 00000100:00100000:0.0:1789762790.582563:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067323648:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762790.582573:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762790.582579:0:7218:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174cd0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c561 expref: 6 pid: 7744 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.582590:0:7218:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174cd0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c561 expref: 6 pid: 7744 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.582619:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7fc6600/0xdf5a7ac59f174cd0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c561 expref: 6 pid: 7744 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.582630:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762790.582637:0:7218:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f174eb3 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174eac expref: 6 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.582646:0:7218:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f174eb3 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f174eac expref: 6 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.582655:0:7218:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f174eb3 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f174eac expref: 6 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.582677:0:7218:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f174eb3 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174eac expref: 6 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.582687:0:7218:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9246680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:7218:1876702074650496:0@lo:105: 00000100:00100000:0.0:1789762790.582713:0:7218:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762790.582740:0:7218:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7fbe600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762790.582746:0:6403:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074650496 00000100:00100000:2.0:1789762790.582751:0:6403:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074650496 00000100:00100000:2.0:1789762790.582777:0:6403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 43 00000100:00100000:2.0:1789762790.582783:0:6403:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7218:x1876702074650496:12345-0@lo:105: 00010000:00010000:2.0:1789762790.582792:0:6403:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.582798:0:6403:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.582804:0:6403:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.582809:0:6403:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.582814:0:6403:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.582819:0:6403:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.582837:0:6403:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c36d1fa7200) 00010000:00010000:0.0:1789762790.582860:0:6420:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.582871:0:6420:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762790.582881:0:6403:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7218:x1876702074650496:12345-0@lo:105: Request processed in 95us (164us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762790.582891:0:6403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 43 00000004:00080000:0.0:1789762790.582893:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1789762790.582912:0:7218:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9246680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:7218:1876702074650496:0@lo:105: 00010000:00010000:0.0:1789762790.582950:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.582959:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762790.582964:0:6420:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.582972:0:6420:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.582983:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9245f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074650624:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1789762790.582994:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067323648:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 431us (517us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.583004:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 67 00000100:00100000:0.0:1789762790.583010:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762790.583035:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7fbee40 going to sleep for 11 seconds 00000100:00100000:2.0:1789762790.583052:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074650624 00000100:00100000:2.0:1789762790.583058:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074650624 00000100:00100000:2.0:1789762790.583091:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 223 00000100:00100000:2.0:1789762790.583100:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9247b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6420:x1876702074650624:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1789762790.583110:0:6423:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762790.583122:0:6423:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174ec1 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f174eba expref: -99 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.583136:0:6423:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174ec1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f174eba expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.583144:0:6423:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174ec1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174eba expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.583149:0:6423:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee100) 00010000:00010000:2.0:1789762790.583151:0:6423:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762790.583153:0:6423:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b11400, rc 0) 00000100:00100000:2.0:1789762790.583182:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9247b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074650624:12345-0@lo:101:mdt00_002.0 Request processed in 85us (172us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762790.583186:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 223 00000100:00100000:0.0:1789762790.583209:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9245f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074650624:0@lo:101:mdt00_002.0 00010000:00010000:0.0:1789762790.583219:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f174ec1 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.583225:0:6420:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174ec1 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762790.583234:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1789762790.583254:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9245f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074650752:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789762790.583274:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762790.583293:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7fbef00 going to sleep for 11 seconds 00000100:00100000:3.0:1789762790.583306:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074650752 00000100:00100000:3.0:1789762790.583312:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074650752 00000100:00100000:3.0:1789762790.583343:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00000100:00100000:3.0:1789762790.583349:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9247480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074650752:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789762790.583402:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9247480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074650752:12345-0@lo:1000:mdt00_002.0 Request processed in 54us (126us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762790.583407:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00100000:0.0:1789762790.583436:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9245f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074650752:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789762790.583474:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9245f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074650880:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789762790.583503:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762790.583523:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7fbef00 going to sleep for 11 seconds 00000100:00100000:1.0:1789762790.583549:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074650880 00000100:00100000:1.0:1789762790.583554:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074650880 00000100:00100000:1.0:1789762790.583579:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:1.0:1789762790.583585:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074650880:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762790.583711:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074650880:12345-0@lo:1000:mdt00_002.0 Request processed in 126us (207us total) trans 0 rc 1/1 00000100:00100000:1.0:1789762790.583716:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00100000:0.0:1789762790.583749:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9245f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074650880:0@lo:1000:mdt00_002.0 00010000:00010000:0.0:1789762790.583783:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 3/0,1 mode: --/EX res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.583798:0:6420:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 3/0,1 mode: --/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.583807:0:6420:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 3/0,1 mode: --/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.583821:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9245f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074651008:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1789762790.583847:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762790.583869:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7fbee40 going to sleep for 11 seconds 00000100:00100000:2.0:1789762790.583875:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074651008 00000100:00100000:2.0:1789762790.583879:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074651008 00000100:00100000:2.0:1789762790.583900:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:2.0:1789762790.583905:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074651008:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1789762790.583911:0:13302:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762790.583950:0:13302:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f174ecf lrc: 2/0,0 mode: --/EX res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f174ec8 expref: -99 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.583972:0:13302:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f174ecf lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f174ec8 expref: 9 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.583979:0:13302:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f174ecf lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174ec8 expref: 9 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.583984:0:13302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37d3beea00) 00010000:00010000:2.0:1789762790.583986:0:13302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762790.583988:0:13302:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b10200, rc 0) 00000100:00100000:2.0:1789762790.584013:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:6420:x1876702074651008:12345-0@lo:101:mdt00_002.0 Request processed in 108us (166us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762790.584017:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00100000:0.0:1789762790.584039:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9245f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074651008:0@lo:101:mdt00_002.0 00010000:00010000:0.0:1789762790.584052:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f174ecf expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.584063:0:6420:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174ecf expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.584082:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174ed6 lrc: 3/0,1 mode: --/EX res: [0x240000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.584091:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174ed6 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.584099:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174ed6 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762790.584602:0:6420:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].6, 424 off36992 00000040:00080000:0.0:1789762790.584648:0:6420:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].6, 504 off37688 00000004:00080000:0.0:1789762790.584655:0:6420:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37f7fbef00 with version 13 00000004:00080000:2.0:1789762790.584686:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 13 rpc_version 13 00000020:00100000:2.0:1789762790.584703:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1789762790.584709:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37d8cced80 x1876702074651264/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1744/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762790.584720:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cced80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074651264:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762790.584742:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762790.584757:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ee88d300 going to sleep for 11 seconds 00000100:00100000:0.0:1789762790.584770:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074651264 00000100:00100000:0.0:1789762790.584781:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074651264 00000100:00100000:0.0:1789762790.584810:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:0.0:1789762790.584817:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7224:x1876702074651264:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762790.585318:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7224:x1876702074651264:12345-0@lo:1000:osp_up0-1.0 Request processed in 500us (572us total) trans 4294967366 rc 0/0 00000100:00100000:0.0:1789762790.585326:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00100000:2.0:1789762790.585388:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cced80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074651264:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1789762790.585402:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xb:0x0]: returning size 16384 00010000:00010000:0.0:1789762790.585411:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174ed6 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585416:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174ed6 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585421:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174ed6 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585424:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762790.585427:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174ed6 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585431:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174ed6 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585437:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:0.0:1789762790.585440:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762790.585444:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174ed6 lrc: 2/0,0 mode: --/EX res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585450:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174ed6 lrc: 0/0,0 mode: --/EX res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585461:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174ecf expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585477:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f174ecf expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585485:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f174ecf expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585504:0:6420:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1789762790.585513:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1789762790.585518:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762790.585520:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.585524:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f174ecf expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585542:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.585542:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6e00 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:2.0:1789762790.585552:0:6408:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f174ecf expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585555:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585574:0:6420:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:2.0:1789762790.585575:0:6408:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f174ecf expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762790.585581:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1789762790.585590:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f174ec1 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585600:0:6420:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f174ec1 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.585602:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762790.585607:0:6408:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 2/0,0 mode: --/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f174ecf expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.585611:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.585627:0:6408:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789762790.585630:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7200 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:1.0:1789762790.585642:0:7217:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762790.585650:0:7217:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.585654:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 1/0,0 mode: --/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f174ecf expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.585658:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:13365:x1876702067323520:12345-192.168.203.37@tcp:36:rm.0 Request processed in 4523us (4587us total) trans 4294967377 rc 0/0 00010000:00010000:2.0:1789762790.585675:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6e00/0xdf5a7ac59f174ec8 lrc: 0/0,0 mode: --/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f174ecf expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762790.585685:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074651392:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762790.585692:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762790.585693:0:7217:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.585694:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 185 00010000:00010000:2.0:1789762790.585695:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762790.585699:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762790.585708:0:7217:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1789762790.585728:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789762790.585734:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762790.585738:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d275b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074651520:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762790.585749:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa7200/0xdf5a7ac59f174eac lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f174eb3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.585761:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762790.585765:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762790.585767:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074651392 00010000:00010000:1.0:1789762790.585768:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762790.585773:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074651392 00000100:00100000:2.0:1789762790.585778:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762790.585810:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074651520 00000100:00100000:0.0:1789762790.585815:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074651520 00000100:00100000:3.0:1789762790.585816:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 28 00000100:00100000:3.0:1789762790.585828:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c91850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3427:x1876702074651392:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789762790.585837:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762790.585847:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f174ecf lrc: 0/0,0 mode: --/EX res: [0x240000402:0xb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f174ec8 expref: 11 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762790.585854:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 29 00010000:00010000:3.0:1789762790.585862:0:9878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37d3beea00) 00000100:00100000:0.0:1789762790.585863:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3424:x1876702074651520:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789762790.585873:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762790.585875:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762790.585887:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11a00/0xdf5a7ac59f174eb3 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f174eac expref: 10 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.585898:0:6406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762790.585900:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762790.585974:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3424:x1876702074651520:12345-0@lo:103:ldlm_bl.0 Request processed in 113us (198us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762790.585982:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 29 00000100:00100000:2.0:1789762790.585990:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d275b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074651520:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789762790.585993:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c91850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+9:3427:x1876702074651392:12345-0@lo:103:ldlm_bl.0 Request processed in 166us (264us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762790.586002:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 28 00000100:00100000:3.0:1789762790.586051:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074651392:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789762790.873753:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.873802:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067323776 00000100:00100000:1.0:1789762790.873814:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067323776 00000100:00100000:1.0:1789762790.873874:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 186 00000100:00100000:1.0:1789762790.873885:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cce300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:13376:x1876702067323776:12345-192.168.203.37@tcp:101:mkdir.0 00010000:00010000:1.0:1789762790.873903:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762790.873960:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95200/0xdf5a7ac59f174edd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c648 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.874026:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94600/0xdf5a7ac59f174ee4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.874048:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94600/0xdf5a7ac59f174ee4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.874058:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94600/0xdf5a7ac59f174ee4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762790.874089:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762790.874092:0:7744:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:1.0:1789762790.874107:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94600/0xdf5a7ac59f174ee4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.874128:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95200/0xdf5a7ac59f174edd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c648 expref: 7 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.874141:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94600/0xdf5a7ac59f174ee4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c648 expref: 6 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.874148:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762790.874151:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762790.874153:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94600, rc 0) 00000100:00100000:1.0:1789762790.874211:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cce300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13376:x1876702067323776:12345-192.168.203.37@tcp:101:mkdir.0 Request processed in 328us (466us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.874217:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 186 00000100:00100000:1.0:1789762790.875414:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.875455:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.875495:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067323904 00000100:00100000:2.0:1789762790.875496:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067324160 00000100:00100000:2.0:1789762790.875506:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067324160 00000100:00100000:3.0:1789762790.875506:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067323904 00000100:00100000:3.0:1789762790.875560:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 187 00000100:00100000:3.0:1789762790.875571:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067323904:12345-192.168.203.37@tcp:41:mkdir.0 00000100:00100000:2.0:1789762790.875580:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 188 00000100:00100000:2.0:1789762790.875597:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13376:x1876702067324160:12345-192.168.203.37@tcp:36:mkdir.0 00000004:00000024:3.0:1789762790.875615:0:11692:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00010000:00010000:2.0:1789762790.875639:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762790.875665:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94600/0xdf5a7ac59f174ee4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c648 expref: 7 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762790.875690:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067323904:12345-192.168.203.37@tcp:41:mkdir.0 Request processed in 119us (275us total) trans 0 rc 0/0 00010000:00010000:2.0:1789762790.875692:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762790.875700:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762790.875704:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 187 00010000:00010000:2.0:1789762790.875779:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174eeb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.875791:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174eeb lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.875798:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174eeb lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.875807:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.875815:0:6419:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174ec1 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f174eba expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.875821:0:6419:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 4/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.875853:0:6419:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174ec1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xdf5a7ac59f174eba expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762790.875868:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cce680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6419:1876702074651648:0@lo:104: 00000100:00100000:2.0:1789762790.875961:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762790.875991:0:8068:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074651648 00000100:00100000:3.0:1789762790.875997:0:8068:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074651648 00000100:00100000:2.0:1789762790.876024:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ee88db40 going to sleep for 11 seconds 00000100:00100000:3.0:1789762790.876048:0:8068:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 44 00000100:00100000:3.0:1789762790.876060:0:8068:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:6419:x1876702074651648:12345-0@lo:104: 00010000:00010000:3.0:1789762790.876072:0:8068:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f174ec1 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.876122:0:8068:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1789762790.876130:0:8068:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:6419:x1876702074651648:12345-0@lo:104: Request processed in 73us (215us total) trans 0 rc 0/0 00010000:00010000:1.0:1789762790.876137:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1789762790.876137:0:8068:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 44 00000100:00100000:2.0:1789762790.876148:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cce680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6419:1876702074651648:0@lo:104: 00010000:00010000:1.0:1789762790.876154:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f174ec1 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.876177:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6200 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:2.0:1789762790.876177:0:6419:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.876184:0:8069:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xdf5a7ac59f174ec1 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762790.876215:0:8069:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xdf5a7ac59f174ec1 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.876243:0:8069:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f174ec1 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.876249:0:8069:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789762790.876275:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f174ec1 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.876281:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36d1fa6200/0xdf5a7ac59f174eba lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f174ec1 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762790.876287:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d275a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074651776:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762790.876321:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762790.876325:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762790.876336:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789762790.876337:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762790.876369:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074651776 00000100:00100000:3.0:1789762790.876375:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074651776 00010000:00010000:3.0:1789762790.876392:0:6404:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174ec1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xdf5a7ac59f174eba expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762790.876417:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 68 00000100:00100000:3.0:1789762790.876428:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cce680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3425:x1876702074651776:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789762790.876438:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:1.0:1789762790.888513:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967366 is committed 00010000:00010000:2.0:1789762790.890735:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174ec1 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xdf5a7ac59f174eba expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.890758:0:6404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee100) 00010000:00010000:2.0:1789762790.890762:0:6404:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.890769:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.890774:0:6404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.890799:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762790.890835:0:6419:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.890847:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762790.890848:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cce680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074651776:12345-0@lo:103:ldlm_bl.0 Request processed in 14423us (14516us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762790.890880:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 68 00000100:00100000:3.0:1789762790.890891:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d275a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074651776:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762790.892259:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174ef9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892276:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174ef9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892287:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174ef9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762790.892319:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000677dd42a reply state 00000000f774d3e4 transno 4294967367 ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174ef9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892333:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174ef9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892343:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:0.0:1789762790.892346:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762790.892349:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762790.892354:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 16384 00010000:00010000:0.0:1789762790.892362:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174eeb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892370:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174eeb lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892377:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174eeb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892384:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174eeb lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892391:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762790.892397:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174eeb lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892405:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174eeb lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892418:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762790.892421:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762790.892435:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174eeb lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892442:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174eeb lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762790.892455:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000677dd42a reply state 00000000f774d3e4 transno 4294967367 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892464:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.892471:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee100) 00010000:00010000:0.0:1789762790.892473:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762790.892476:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:0.0:1789762790.892555:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13376:x1876702067324160:12345-192.168.203.37@tcp:36:mkdir.0 Request processed in 16964us (17096us total) trans 4294967367 rc 0/0 00000100:00100000:0.0:1789762790.892562:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 188 00000100:00100000:2.0:1789762790.897776:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.897828:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067324288 00000100:00100000:0.0:1789762790.897835:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067324288 00000100:00100000:0.0:1789762790.897874:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 189 00000100:00100000:0.0:1789762790.897881:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13377:x1876702067324288:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762790.897898:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762790.897909:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174f00 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c64f expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.898010:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6600/0xdf5a7ac59f174f07 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.898022:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6600/0xdf5a7ac59f174f07 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.898045:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6600/0xdf5a7ac59f174f07 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762790.898069:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762790.898073:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762790.898089:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6600/0xdf5a7ac59f174f07 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.898111:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174f00 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c64f expref: 9 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.898123:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6600/0xdf5a7ac59f174f07 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c64f expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.898129:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762790.898131:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762790.898133:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa6600, rc 0) 00000100:00100000:0.0:1789762790.898169:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13377:x1876702067324288:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 287us (395us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762790.898175:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 189 00000100:00100000:3.0:1789762790.899150:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.899192:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067324544 00000100:00100000:2.0:1789762790.899200:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067324544 00000100:00100000:2.0:1789762790.899251:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 190 00000100:00100000:2.0:1789762790.899261:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13377:x1876702067324544:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762790.899276:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762790.899279:0:13389:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1789762790.899296:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6600/0xdf5a7ac59f174f07 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c64f expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899317:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762790.899320:0:13389:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762790.899330:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11e00/0xdf5a7ac59f174f0e lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c656 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899436:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f15 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899446:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f15 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899453:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f15 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899461:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174f1c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899467:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174f1c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899472:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174f1c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899523:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10600/0xdf5a7ac59f174f23 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899528:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10600/0xdf5a7ac59f174f23 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899532:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10600/0xdf5a7ac59f174f23 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762790.899546:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 16384 00010000:00010000:2.0:1789762790.899559:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10600/0xdf5a7ac59f174f23 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899564:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10600/0xdf5a7ac59f174f23 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899569:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10600/0xdf5a7ac59f174f23 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899574:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762790.899579:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10600/0xdf5a7ac59f174f23 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899587:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10600/0xdf5a7ac59f174f23 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899598:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:2.0:1789762790.899602:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762790.899606:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10600/0xdf5a7ac59f174f23 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899615:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10600/0xdf5a7ac59f174f23 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899633:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f15 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899640:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f15 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899644:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f15 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899655:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f15 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899660:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762790.899662:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f15 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899667:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f15 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899675:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762790.899677:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762790.899679:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f15 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899684:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f15 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899691:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174f1c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899695:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174f1c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899699:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174f1c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899703:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762790.899705:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174f1c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899717:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174f1c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899724:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee100) 00010000:00010000:2.0:1789762790.899726:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762790.899728:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174f1c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899732:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10c00/0xdf5a7ac59f174f1c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899754:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11e00/0xdf5a7ac59f174f0e lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c656 expref: 8 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899761:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762790.899763:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762790.899765:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11e00/0xdf5a7ac59f174f0e lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c656 expref: 8 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.899770:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b11e00, rc 0) 00000100:00100000:2.0:1789762790.899811:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13377:x1876702067324544:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 552us (663us total) trans 0 rc 301/301 00000100:00100000:2.0:1789762790.899816:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 190 00000100:00100000:2.0:1789762790.900658:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.900702:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067324672 00000100:00100000:3.0:1789762790.900709:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067324672 00000100:00100000:3.0:1789762790.900760:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 191 00000100:00100000:3.0:1789762790.900774:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13377:x1876702067324672:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:3.0:1789762790.900794:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762790.900806:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f174f2a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c65d expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.900849:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174f31 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.900861:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174f31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.900873:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174f31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762790.900893:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762790.900896:0:11692:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762790.900909:0:11692:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174f31 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.900967:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f174f2a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c65d expref: 9 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.900988:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174f31 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c65d expref: 8 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.900995:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762790.900998:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762790.901000:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff934e00, rc 0) 00000100:00100000:3.0:1789762790.901038:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13377:x1876702067324672:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 270us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762790.901044:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 191 00000100:00100000:3.0:1789762790.901988:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.902029:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067324928 00000100:00100000:0.0:1789762790.902039:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067324928 00000100:00100000:0.0:1789762790.902096:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 192 00000100:00100000:0.0:1789762790.902106:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13377:x1876702067324928:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762790.902120:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762790.902132:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7000/0xdf5a7ac59f174f38 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c664 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902210:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174f3f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902220:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174f3f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902227:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174f3f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902235:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f174f46 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902240:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f174f46 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902246:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f174f46 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762790.902358:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 16384 00010000:00010000:0.0:1789762790.902451:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174f3f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902461:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174f3f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902467:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174f3f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902475:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174f3f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902481:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762790.902494:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174f3f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902503:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174f3f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902517:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762790.902520:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762790.902525:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174f3f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902533:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7c00/0xdf5a7ac59f174f3f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902549:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f174f46 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902556:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f174f46 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902561:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f174f46 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902577:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762790.902580:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f174f46 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902587:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f174f46 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902598:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee100) 00010000:00010000:0.0:1789762790.902601:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762790.902613:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f174f46 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902621:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7800/0xdf5a7ac59f174f46 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902659:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7000/0xdf5a7ac59f174f38 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c664 expref: 10 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902671:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762790.902675:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762790.902679:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7000/0xdf5a7ac59f174f38 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c664 expref: 10 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.902689:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa7000, rc 0) 00000100:00100000:0.0:1789762790.902743:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:13377:x1876702067324928:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 638us (754us total) trans 4294967368 rc 301/301 00000100:00100000:0.0:1789762790.902752:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 192 00000100:00100000:0.0:1789762790.903826:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.903865:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067325184 00000100:00100000:1.0:1789762790.903871:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067325184 00000100:00100000:1.0:1789762790.903905:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 193 00000100:00100000:1.0:1789762790.903935:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:13377:x1876702067325184:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:1.0:1789762790.903982:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174f54 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.904006:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174f54 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.904014:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174f54 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762790.904133:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009caa208d reply state 000000002d18d228 transno 4294967369 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174f54 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.904143:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174f54 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.904148:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:1.0:1789762790.904150:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:1.0:1789762790.904152:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:1.0:1789762790.904156:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 16384 00000100:00100000:1.0:1789762790.904204:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:13377:x1876702067325184:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 291us (378us total) trans 4294967369 rc 0/0 00000100:00100000:1.0:1789762790.904210:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 193 00000100:00100000:2.0:1789762790.904818:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.904873:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067325440 00000100:00100000:0.0:1789762790.904880:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067325440 00000100:00100000:0.0:1789762790.904964:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 48 00000100:00100000:0.0:1789762790.904972:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2733480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:13377:x1876702067325440:12345-192.168.203.37@tcp:35:lfs.0 00000100:00100000:0.0:1789762790.905106:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2733480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:13377:x1876702067325440:12345-192.168.203.37@tcp:35:lfs.0 Request processed in 135us (290us total) trans 4294967370 rc 0/0 00000100:00100000:0.0:1789762790.905112:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 48 00000100:00100000:2.0:1789762790.908555:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762790.908605:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067325568 00000100:00100000:0.0:1789762790.908612:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067325568 00000100:00100000:0.0:1789762790.908659:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 194 00000100:00100000:0.0:1789762790.908669:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2730a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:13378:x1876702067325568:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:0.0:1789762790.908684:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762790.908696:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174f5b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c66b expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908735:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174f62 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908746:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174f62 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908755:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174f62 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908766:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174f69 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908774:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174f69 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908782:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174f69 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908839:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174f70 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908850:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174f70 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908859:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174f70 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762790.908890:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 16384 00000004:00200000:0.0:1789762790.908894:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762790.908907:0:6419:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174f70 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908959:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174f62 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908968:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174f62 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908978:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174f62 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908988:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174f62 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.908994:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762790.908997:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174f62 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909002:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174f62 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909013:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762790.909015:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762790.909017:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174f62 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909022:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6200/0xdf5a7ac59f174f62 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909039:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174f69 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909045:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174f69 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909049:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174f69 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909055:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762790.909059:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174f69 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909064:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174f69 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909076:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee100) 00010000:00010000:0.0:1789762790.909080:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762790.909082:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174f69 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909086:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6800/0xdf5a7ac59f174f69 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909105:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6c00/0xdf5a7ac59f174f5b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c66b expref: 14 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909114:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174f70 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c66b expref: 13 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762790.909127:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:0.0:1789762790.909128:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762790.909130:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa7600, rc 0) 00000100:00100000:0.0:1789762790.909162:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2730a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:13378:x1876702067325568:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 496us (610us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762790.909168:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 194 00000100:00100000:0.0:1789762790.910293:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.910348:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067325824 00000100:00100000:2.0:1789762790.910357:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067325824 00000100:00100000:2.0:1789762790.910428:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 195 00000100:00100000:2.0:1789762790.910444:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9247100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:13378:x1876702067325824:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:2.0:1789762790.910481:0:6418:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762790.910506:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa7600/0xdf5a7ac59f174f70 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c66b expref: 13 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.910526:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:2.0:1789762790.910530:0:6418:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762790.910550:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f77 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.910564:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f77 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.910575:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f77 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762790.910824:0:6418:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002b749c01 reply state 000000006654836c transno 4294967371 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f77 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.910835:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f77 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.910841:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:2.0:1789762790.910844:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:2.0:1789762790.910847:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762790.910853:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 16384 00000100:00100000:2.0:1789762790.910970:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9247100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13378:x1876702067325824:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 474us (622us total) trans 4294967371 rc 0/0 00000100:00100000:2.0:1789762790.910978:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 195 00000100:00100000:3.0:1789762790.912101:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d98a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074651904:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762790.912154:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762790.912220:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074651904 00000100:00100000:0.0:1789762790.912227:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074651904 00000100:00100000:0.0:1789762790.912263:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:0.0:1789762790.912270:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074651904:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762790.912290:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762790.912345:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074651904:12345-0@lo:41:osp-pre-0-1.0 Request processed in 74us (188us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762790.912350:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:0.0:1789762790.912392:0:3426:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967366 gen 2 00000004:00080000:0.0:1789762790.912398:0:3426:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967366 committed transno 4294967366 00000100:00100000:0.0:1789762790.912434:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d98a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074651904:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762790.929030:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762790.929323:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067325952 00000100:00100000:1.0:1789762790.929331:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067325952 00000100:00100000:1.0:1789762790.929367:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 196 00000100:00100000:1.0:1789762790.929376:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13384:x1876702067325952:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:1.0:1789762790.929390:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762790.929403:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f174f7e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c672 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929447:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f174f85 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929465:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f174f85 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929474:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f174f85 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929482:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95e00/0xdf5a7ac59f174f8c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929487:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95e00/0xdf5a7ac59f174f8c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929491:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95e00/0xdf5a7ac59f174f8c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929556:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94000/0xdf5a7ac59f174f93 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929561:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94000/0xdf5a7ac59f174f93 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929565:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94000/0xdf5a7ac59f174f93 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762790.929578:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 16384 00000004:00200000:1.0:1789762790.929580:0:13389:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:1.0:1789762790.929592:0:13389:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94000/0xdf5a7ac59f174f93 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929599:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f174f85 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929612:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f174f85 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929616:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f174f85 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929620:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f174f85 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929624:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762790.929627:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f174f85 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929632:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f174f85 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929641:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762790.929644:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762790.929646:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f174f85 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929651:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f174f85 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929658:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95e00/0xdf5a7ac59f174f8c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929662:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95e00/0xdf5a7ac59f174f8c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929665:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95e00/0xdf5a7ac59f174f8c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929672:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762790.929674:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95e00/0xdf5a7ac59f174f8c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929678:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95e00/0xdf5a7ac59f174f8c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929682:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee100) 00010000:00010000:1.0:1789762790.929684:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762790.929686:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95e00/0xdf5a7ac59f174f8c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929689:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95e00/0xdf5a7ac59f174f8c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929709:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f174f7e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c672 expref: 16 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929722:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94000/0xdf5a7ac59f174f93 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c672 expref: 15 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762790.929729:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:1.0:1789762790.929732:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1789762790.929734:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94000, rc 0) 00000100:00100000:1.0:1789762790.929782:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:13384:x1876702067325952:12345-192.168.203.37@tcp:101:dd.500 Request processed in 408us (760us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762790.929789:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 196 00000100:00100000:2.0:1789762790.930844:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762790.930888:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067326208 00000100:00100000:3.0:1789762790.930896:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067326208 00000100:00100000:3.0:1789762790.930979:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 197 00000100:00100000:3.0:1789762790.930989:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d98a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:13384:x1876702067326208:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762790.931004:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762790.931019:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f174f9a lrc: 2/0,0 mode: --/CW res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c679 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931126:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934400/0xdf5a7ac59f174fa1 lrc: 3/0,1 mode: --/CW res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931138:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934400/0xdf5a7ac59f174fa1 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931145:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934400/0xdf5a7ac59f174fa1 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931157:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174fa8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xe:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931162:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174fa8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931167:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174fa8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762790.931642:0:11692:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x5:0x0], index 0: 0 00010000:00010000:3.0:1789762790.931788:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934600/0xdf5a7ac59f174faf lrc: 3/1,0 mode: --/PR res: [0x200000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931795:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934600/0xdf5a7ac59f174faf lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931800:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934600/0xdf5a7ac59f174faf lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931826:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934400/0xdf5a7ac59f174fa1 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931832:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934400/0xdf5a7ac59f174fa1 lrc: 2/0,0 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931837:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934400/0xdf5a7ac59f174fa1 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931841:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762790.931843:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934400/0xdf5a7ac59f174fa1 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931848:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934400/0xdf5a7ac59f174fa1 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931856:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:3.0:1789762790.931858:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762790.931861:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934400/0xdf5a7ac59f174fa1 lrc: 2/0,0 mode: --/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931866:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934400/0xdf5a7ac59f174fa1 lrc: 0/0,0 mode: --/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762790.931883:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004552665b reply state 00000000f2899049 transno 4294967372 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174fa8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931893:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174fa8 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931897:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x4009f790 (ffff9c37c72ba300) 00010000:00010000:3.0:1789762790.931899:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1789762790.931901:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789762790.931968:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935e00/0xdf5a7ac59f174f9a lrc: 0/0,0 mode: --/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c679 expref: 18 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931985:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934600/0xdf5a7ac59f174faf lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c679 expref: 17 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.931991:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff9c37c72bae00) 00010000:00010000:3.0:1789762790.931994:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762790.931997:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ff934600, rc 0) 00000100:00100000:3.0:1789762790.932048:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d98a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13384:x1876702067326208:12345-192.168.203.37@tcp:101:dd.500 Request processed in 1062us (1208us total) trans 4294967372 rc 0/0 00000100:00100000:3.0:1789762790.932055:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 197 00000100:00100000:0.0:1789762790.936577:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.936638:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067326336 00000100:00100000:2.0:1789762790.936647:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067326336 00000100:00100000:2.0:1789762790.936702:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263678497219 time 263678599199 state 0x0 ref 1 00000100:00100000:2.0:1789762790.936715:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 263728599199 time 263678599199 state 0x1 ref 1 00000100:00100000:2.0:1789762790.936738:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 2 deadline 263728599199 time 264183816517 state 0x0 ref 1 00000100:00100000:2.0:1789762790.936749:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2395:x1876702067326336:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762790.936796:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789762790.936803:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f174fbd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.936816:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:2.0:1789762790.936820:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762790.936824:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f174fbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.936836:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f174fbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762790.967263:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967571 is committed 00002000:00010000:3.0:1789762790.967291:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:3.0:1789762790.967298:0:8059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb mtime from disk: 0 -> 1789762790 00002000:00010000:3.0:1789762790.967300:0:8059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb ctime from disk: 0 -> 1789762790 00002000:00010000:3.0:1789762790.967302:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 1 -> 2051 00000001:00080000:0.0:1789762790.967320:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967572 is committed 00010000:00010000:3.0:1789762790.967320:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f174fbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.967348:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f174fbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.967356:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f174fbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.967361:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762790.967366:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f174fbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.967373:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f174fbd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.967393:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f174fbd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.967398:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f174fbd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762790.967512:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067326336:12345-192.168.203.37@tcp:4:dd.500 Request processed in 30757us (30948us total) trans 4294967572 rc 0/0 00000100:00100000:3.0:1789762790.967523:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 31 00000100:00100000:3.0:1789762790.967535:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 2 deadline 263728599199 time 264183816517 state 0x2 ref 0 00000100:00100000:0.0:1789762790.980317:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.980374:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067326464 00000100:00100000:2.0:1789762790.980386:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067326464 00000100:00100000:2.0:1789762790.980435:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 2 deadline 263728599199 time 264183816517 state 0x0 ref 1 00000100:00100000:2.0:1789762790.980457:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 2 deadline 264233816517 time 264183816517 state 0x1 ref 1 00000100:00100000:2.0:1789762790.980492:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 1 deadline 264233816517 time 264227568031 state 0x0 ref 1 00000100:00100000:2.0:1789762790.980507:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067326464:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762790.980554:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 1048576, mtime: 0x6aad9ce6, atime: 0x0, ctime: 0x6aad9ce6, blocks: 0x803 00010000:00010000:2.0:1789762790.980561:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f174fc4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.980574:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:2.0:1789762790.980577:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762790.980582:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f174fc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.980593:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f174fc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762790.993516:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967573 is committed 00002000:00010000:3.0:1789762790.993565:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:3.0:1789762790.993573:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:3.0:1789762790.993592:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f174fc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.993603:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f174fc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.993610:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f174fc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.993616:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762790.993619:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f174fc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.993626:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f174fc4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.993643:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f174fc4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762790.993648:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f174fc4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762790.993733:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067326464:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13225us (13419us total) trans 4294967573 rc 0/0 00000100:00100000:3.0:1789762790.993744:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 32 00000100:00100000:3.0:1789762790.993754:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 1 deadline 264233816517 time 264227568031 state 0x2 ref 0 00000100:00100000:0.0:1789762790.996012:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762790.996065:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067326592 00000100:00100000:2.0:1789762790.996072:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067326592 00000100:00100000:2.0:1789762790.996105:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 1 deadline 264233816517 time 264227568031 state 0x0 ref 1 00000100:00100000:2.0:1789762790.996115:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 1 deadline 264277568031 time 264227568031 state 0x1 ref 1 00000100:00100000:2.0:1789762790.996135:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264277568031 time 264243213887 state 0x0 ref 1 00000100:00100000:2.0:1789762790.996152:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067326592:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762790.996180:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 2097152, mtime: 0x6aad9ce6, atime: 0x0, ctime: 0x6aad9ce6, blocks: 0x1007 00010000:00010000:2.0:1789762790.996187:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174fcb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.996198:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:2.0:1789762790.996201:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762790.996206:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174fcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762790.996214:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174fcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.009134:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967574 is committed 00002000:00010000:3.0:1789762791.009184:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:3.0:1789762791.009191:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:3.0:1789762791.009201:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174fcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.009211:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174fcb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.009218:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174fcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.009223:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762791.009226:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174fcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.009233:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174fcb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.009247:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174fcb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.009252:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021abf560/0xdf5a7ac59f174fcb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.009319:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067326592:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13166us (13309us total) trans 4294967574 rc 0/0 00000100:00100000:3.0:1789762791.009336:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 33 00000100:00100000:3.0:1789762791.009346:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264277568031 time 264243213887 state 0x2 ref 0 00000100:00100000:1.0:1789762791.011362:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762791.011403:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067326720 00000100:00100000:2.0:1789762791.011409:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067326720 00000100:00080000:1.0:1789762791.011442:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762791.011451:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264277568031 time 264243213887 state 0x0 ref 1 00000100:00100000:2.0:1789762791.011461:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264293213887 time 264243213887 state 0x1 ref 1 00010000:00010000:3.0:1789762791.040057:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762791.046207:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264293213887 time 264293272966 state 0x0 ref 1 00000100:00100000:3.0:1789762791.046235:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067326720:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762791.046290:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 3145728, mtime: 0x6aad9ce6, atime: 0x0, ctime: 0x6aad9ce6, blocks: 0x1807 00010000:00010000:3.0:1789762791.046297:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fd2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.046311:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:3.0:1789762791.046315:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762791.046320:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.046331:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.059847:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967575 is committed 00002000:00010000:3.0:1789762791.059872:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:3.0:1789762791.059877:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:3.0:1789762791.059887:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.059897:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.059939:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.059946:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762791.059950:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.059957:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.059973:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fd2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.059978:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.060048:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067326720:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13820us (48686us total) trans 4294967575 rc 0/0 00000100:00100000:3.0:1789762791.060058:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 34 00000100:00100000:3.0:1789762791.060068:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264293213887 time 264293272966 state 0x2 ref 0 00000100:00100000:1.0:1789762791.062230:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762791.062309:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067326848 00000100:00100000:0.0:1789762791.062323:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067326848 00000100:00100000:0.0:1789762791.062363:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264293213887 time 264293272966 state 0x0 ref 1 00000100:00100000:0.0:1789762791.062374:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264343272966 time 264293272966 state 0x1 ref 1 00000100:00100000:0.0:1789762791.096266:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264343272966 time 264343334847 state 0x0 ref 1 00000100:00100000:0.0:1789762791.096285:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067326848:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762791.096325:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 4194304, mtime: 0x6aad9ce6, atime: 0x0, ctime: 0x6aad9ce6, blocks: 0x2007 00010000:00010000:0.0:1789762791.096332:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174fd9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.096344:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:0.0:1789762791.096348:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762791.096352:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174fd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.096363:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174fd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.107297:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967576 is committed 00002000:00010000:3.0:1789762791.107302:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:3.0:1789762791.107307:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:3.0:1789762791.107320:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174fd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.107349:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174fd9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.107358:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174fd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.107367:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762791.107370:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174fd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.107377:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174fd9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.107393:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174fd9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.107398:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eaf53dc1/0xdf5a7ac59f174fd9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.107475:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067326848:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11192us (45241us total) trans 4294967576 rc 0/0 00000100:00100000:3.0:1789762791.107488:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 35 00000100:00100000:3.0:1789762791.107500:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264343272966 time 264343334847 state 0x2 ref 0 00000100:00100000:2.0:1789762791.109765:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.109798:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067326976 00000100:00100000:3.0:1789762791.109813:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067326976 00000100:00100000:3.0:1789762791.109851:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264343272966 time 264343334847 state 0x0 ref 1 00000100:00100000:3.0:1789762791.109862:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264393334847 time 264343334847 state 0x1 ref 1 00000100:00100000:2.0:1789762791.146373:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264393334847 time 264393437965 state 0x0 ref 1 00000100:00100000:2.0:1789762791.146397:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067326976:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762791.146445:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 5242880, mtime: 0x6aad9ce6, atime: 0x0, ctime: 0x6aad9ce6, blocks: 0x2807 00010000:00010000:2.0:1789762791.146463:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f174fe0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.146476:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:2.0:1789762791.146480:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762791.146492:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f174fe0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.146504:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f174fe0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.158319:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967577 is committed 00002000:00010000:2.0:1789762791.158321:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:2.0:1789762791.158328:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:2.0:1789762791.158339:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f174fe0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.158349:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f174fe0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.158355:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f174fe0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.158360:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762791.158364:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f174fe0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.158371:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f174fe0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.158395:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f174fe0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.158400:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f174fe0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762791.158470:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067326976:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12081us (48704us total) trans 4294967577 rc 0/0 00000100:00100000:2.0:1789762791.158478:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 36 00000100:00100000:2.0:1789762791.158488:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264393334847 time 264393437965 state 0x2 ref 0 00000100:00100000:0.0:1789762791.160958:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.161012:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067327104 00000100:00100000:3.0:1789762791.161022:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067327104 00000100:00100000:3.0:1789762791.161071:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264393334847 time 264393437965 state 0x0 ref 1 00000100:00100000:3.0:1789762791.161087:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264443437965 time 264393437965 state 0x1 ref 1 00000100:00100000:3.0:1789762791.196439:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264443437965 time 264443500024 state 0x0 ref 1 00000100:00100000:3.0:1789762791.196469:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067327104:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762791.196528:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 6291456, mtime: 0x6aad9ce6, atime: 0x0, ctime: 0x6aad9ce6, blocks: 0x3007 00010000:00010000:3.0:1789762791.196535:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fe7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.196549:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:3.0:1789762791.196552:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762791.196557:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fe7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.196568:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fe7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.208083:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967578 is committed 00002000:00010000:3.0:1789762791.208102:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:3.0:1789762791.208107:0:14171:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb mtime from disk: 1789762790 -> 1789762791 00002000:00010000:3.0:1789762791.208110:0:14171:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb ctime from disk: 1789762790 -> 1789762791 00002000:00010000:3.0:1789762791.208113:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 12295 -> 14343 00010000:00010000:3.0:1789762791.208125:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fe7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.208138:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fe7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.208157:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fe7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.208165:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762791.208171:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fe7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.208181:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fe7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.208200:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fe7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.208208:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f174fe7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.208299:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067327104:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11839us (47338us total) trans 4294967578 rc 0/0 00000100:00100000:3.0:1789762791.208307:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 37 00000100:00100000:3.0:1789762791.208315:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264443437965 time 264443500024 state 0x2 ref 0 00000100:00100000:1.0:1789762791.211245:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.211281:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067327232 00000100:00100000:3.0:1789762791.211289:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067327232 00000100:00100000:3.0:1789762791.211332:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264443437965 time 264443500024 state 0x0 ref 1 00000100:00100000:3.0:1789762791.211345:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264493500024 time 264443500024 state 0x1 ref 1 00000100:00100000:1.0:1789762791.246533:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264493500024 time 264493593553 state 0x0 ref 1 00000100:00100000:1.0:1789762791.246561:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067327232:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762791.246618:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 7340032, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x3807 00010000:00010000:1.0:1789762791.246628:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174fee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.246645:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:1.0:1789762791.246650:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762791.246657:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174fee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.246673:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174fee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.258625:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967579 is committed 00002000:00010000:3.0:1789762791.258667:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:3.0:1789762791.258672:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:3.0:1789762791.258683:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174fee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.258697:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174fee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.258708:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174fee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.258716:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762791.258722:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174fee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.258747:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174fee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.258769:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174fee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.258777:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f174fee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.258880:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067327232:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12327us (47631us total) trans 4294967579 rc 0/0 00000100:00100000:3.0:1789762791.258892:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 38 00000100:00100000:3.0:1789762791.258905:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264493500024 time 264493593553 state 0x2 ref 0 00000100:00100000:1.0:1789762791.261480:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762791.261559:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067327360 00000100:00100000:2.0:1789762791.261568:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067327360 00000100:00100000:2.0:1789762791.261608:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264493500024 time 264493593553 state 0x0 ref 1 00000100:00100000:2.0:1789762791.261619:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264543593553 time 264493593553 state 0x1 ref 1 00000100:00100000:0.0:1789762791.296617:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264543593553 time 264543684137 state 0x0 ref 1 00000100:00100000:0.0:1789762791.296636:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067327360:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762791.296680:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 8388608, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x4007 00010000:00010000:0.0:1789762791.296698:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000024ea87a4/0xdf5a7ac59f174ff5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.296712:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:0.0:1789762791.296716:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762791.296721:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024ea87a4/0xdf5a7ac59f174ff5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.296731:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024ea87a4/0xdf5a7ac59f174ff5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.308432:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967580 is committed 00002000:00010000:3.0:1789762791.308435:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 8388608 -> 9437184 00002000:00010000:3.0:1789762791.308440:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 16391 -> 18439 00010000:00010000:3.0:1789762791.308452:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000024ea87a4/0xdf5a7ac59f174ff5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.308462:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000024ea87a4/0xdf5a7ac59f174ff5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.308471:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000024ea87a4/0xdf5a7ac59f174ff5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.308493:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762791.308499:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000024ea87a4/0xdf5a7ac59f174ff5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.308509:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000024ea87a4/0xdf5a7ac59f174ff5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.308530:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000024ea87a4/0xdf5a7ac59f174ff5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.308539:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024ea87a4/0xdf5a7ac59f174ff5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.308621:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067327360:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11986us (47137us total) trans 4294967580 rc 0/0 00000100:00100000:3.0:1789762791.308633:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 39 00000100:00100000:3.0:1789762791.308645:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264543593553 time 264543684137 state 0x2 ref 0 00000100:00100000:2.0:1789762791.311177:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.311203:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067327488 00000100:00100000:3.0:1789762791.311212:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067327488 00000100:00100000:3.0:1789762791.311252:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264543593553 time 264543684137 state 0x0 ref 1 00000100:00100000:3.0:1789762791.311263:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264593684137 time 264543684137 state 0x1 ref 1 00000100:00100000:2.0:1789762791.346743:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264593684137 time 264593809720 state 0x0 ref 1 00000100:00100000:2.0:1789762791.346767:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067327488:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762791.346814:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 9437184, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x4807 00010000:00010000:2.0:1789762791.346821:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174ffc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.346855:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:2.0:1789762791.346862:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762791.346867:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174ffc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.346878:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174ffc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762791.358331:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967581 is committed 00002000:00010000:3.0:1789762791.358354:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 9437184 -> 10485760 00002000:00010000:3.0:1789762791.358366:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 18439 -> 20487 00010000:00010000:3.0:1789762791.358376:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174ffc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.358386:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174ffc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.358393:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174ffc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.358397:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762791.358401:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174ffc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.358408:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174ffc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.358423:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174ffc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.358428:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f174ffc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.358496:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067327488:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11737us (47317us total) trans 4294967581 rc 0/0 00000100:00100000:3.0:1789762791.358505:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 40 00000100:00100000:3.0:1789762791.358515:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264593684137 time 264593809720 state 0x2 ref 0 00000100:00100000:0.0:1789762791.360992:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.361033:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067327616 00000100:00100000:3.0:1789762791.361041:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067327616 00000100:00100000:3.0:1789762791.361079:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264593684137 time 264593809720 state 0x0 ref 1 00000100:00100000:3.0:1789762791.361090:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264643809720 time 264593809720 state 0x1 ref 1 00000100:00100000:3.0:1789762791.396784:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264643809720 time 264643852671 state 0x0 ref 1 00000100:00100000:3.0:1789762791.396805:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067327616:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762791.396853:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 10485760, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x5007 00010000:00010000:3.0:1789762791.396862:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175003 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.396879:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:3.0:1789762791.396884:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762791.396891:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175003 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.396902:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175003 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.407258:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967582 is committed 00002000:00010000:3.0:1789762791.407282:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:3.0:1789762791.407288:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:3.0:1789762791.407298:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175003 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.407318:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175003 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.407324:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175003 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.407329:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762791.407332:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175003 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.407338:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175003 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.407353:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175003 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.407361:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175003 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.407439:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067327616:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10639us (46445us total) trans 4294967582 rc 0/0 00000100:00100000:3.0:1789762791.407447:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 41 00000100:00100000:3.0:1789762791.407456:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264643809720 time 264643852671 state 0x2 ref 0 00000100:00100000:2.0:1789762791.409667:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.409699:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067327744 00000100:00100000:3.0:1789762791.409711:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067327744 00000100:00100000:3.0:1789762791.409746:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264643809720 time 264643852671 state 0x0 ref 1 00000100:00100000:3.0:1789762791.409756:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264693852671 time 264643852671 state 0x1 ref 1 00000100:00100000:1.0:1789762791.446892:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264693852671 time 264693960635 state 0x0 ref 1 00000100:00100000:1.0:1789762791.446948:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067327744:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762791.446998:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 11534336, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x5807 00010000:00010000:1.0:1789762791.447005:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17500a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.447018:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:1.0:1789762791.447022:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762791.447026:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17500a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.447037:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17500a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.458977:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967583 is committed 00002000:00010000:3.0:1789762791.459011:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 11534336 -> 12582912 00002000:00010000:3.0:1789762791.459017:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 22535 -> 24583 00010000:00010000:3.0:1789762791.459029:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17500a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.459040:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17500a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.459046:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17500a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.459052:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762791.459055:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17500a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.459061:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17500a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.459077:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17500a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.459081:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17500a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.459147:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067327744:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12238us (49479us total) trans 4294967583 rc 0/0 00000100:00100000:3.0:1789762791.459156:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 42 00000100:00100000:3.0:1789762791.459165:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264693852671 time 264693960635 state 0x2 ref 0 00000100:00100000:1.0:1789762791.461638:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.461672:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067327872 00000100:00100000:3.0:1789762791.461680:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067327872 00000100:00100000:3.0:1789762791.461718:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264693852671 time 264693960635 state 0x0 ref 1 00000100:00100000:3.0:1789762791.461729:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264743960635 time 264693960635 state 0x1 ref 1 00000100:00100000:2.0:1789762791.496988:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264743960635 time 264744053896 state 0x0 ref 1 00000100:00100000:2.0:1789762791.497013:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067327872:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762791.497061:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 12582912, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x6007 00010000:00010000:2.0:1789762791.497069:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175011 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.497081:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:2.0:1789762791.497085:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762791.497089:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175011 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.497100:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175011 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.509240:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967584 is committed 00002000:00010000:2.0:1789762791.509269:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 12582912 -> 13631488 00002000:00010000:2.0:1789762791.509281:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 24583 -> 26631 00010000:00010000:2.0:1789762791.509292:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175011 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.509305:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175011 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.509316:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175011 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.509324:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762791.509330:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175011 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.509340:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175011 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.509360:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175011 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.509366:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175011 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762791.509445:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067327872:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12440us (47804us total) trans 4294967584 rc 0/0 00000100:00100000:2.0:1789762791.509454:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 43 00000100:00100000:2.0:1789762791.509464:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264743960635 time 264744053896 state 0x2 ref 0 00000100:00100000:1.0:1789762791.512049:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.512090:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067328000 00000100:00100000:3.0:1789762791.512098:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067328000 00000100:00100000:3.0:1789762791.512137:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264743960635 time 264744053896 state 0x0 ref 1 00000100:00100000:3.0:1789762791.512147:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264794053896 time 264744053896 state 0x1 ref 1 00000100:00100000:2.0:1789762791.547075:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264794053896 time 264794142118 state 0x0 ref 1 00000100:00100000:2.0:1789762791.547099:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067328000:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762791.547155:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 13631488, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x6807 00010000:00010000:2.0:1789762791.547162:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175018 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.547175:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:2.0:1789762791.547179:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762791.547183:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175018 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.547195:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175018 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.558300:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967585 is committed 00002000:00010000:2.0:1789762791.558304:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 13631488 -> 14680064 00002000:00010000:2.0:1789762791.558310:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:2.0:1789762791.558321:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175018 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.558331:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175018 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.558338:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175018 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.558354:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762791.558357:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175018 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.558364:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175018 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.558378:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175018 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.558384:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175018 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762791.558445:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067328000:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11354us (46395us total) trans 4294967585 rc 0/0 00000100:00100000:2.0:1789762791.558454:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 44 00000100:00100000:2.0:1789762791.558464:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264794053896 time 264794142118 state 0x2 ref 0 00000100:00100000:0.0:1789762791.561072:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.561117:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067328128 00000100:00100000:3.0:1789762791.561125:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067328128 00000100:00100000:3.0:1789762791.561165:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264794053896 time 264794142118 state 0x0 ref 1 00000100:00100000:3.0:1789762791.561176:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264844142118 time 264794142118 state 0x1 ref 1 00000100:00100000:1.0:1789762791.597152:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264844142118 time 264844219324 state 0x0 ref 1 00000100:00100000:1.0:1789762791.597177:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067328128:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762791.597224:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 14680064, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x7007 00010000:00010000:1.0:1789762791.597231:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17501f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.597254:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:1.0:1789762791.597258:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762791.597262:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17501f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.597274:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17501f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.608884:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967586 is committed 00002000:00010000:3.0:1789762791.608953:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 14680064 -> 15728640 00002000:00010000:3.0:1789762791.608964:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:3.0:1789762791.608975:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17501f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.608985:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17501f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.608991:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17501f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.608996:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762791.609000:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17501f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.609006:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17501f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.609021:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17501f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.609026:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17501f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.609090:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067328128:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11922us (48019us total) trans 4294967586 rc 0/0 00000100:00100000:3.0:1789762791.609098:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 45 00000100:00100000:3.0:1789762791.609107:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264844142118 time 264844219324 state 0x2 ref 0 00000100:00100000:0.0:1789762791.611661:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762791.611713:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067328256 00000100:00100000:2.0:1789762791.611722:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067328256 00000100:00100000:2.0:1789762791.611762:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264844142118 time 264844219324 state 0x0 ref 1 00000100:00100000:2.0:1789762791.611773:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264894219324 time 264844219324 state 0x1 ref 1 00000100:00100000:1.0:1789762791.647250:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264894219324 time 264894313805 state 0x0 ref 1 00000100:00100000:1.0:1789762791.647275:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067328256:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762791.647326:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 15728640, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x7807 00010000:00010000:1.0:1789762791.647333:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175026 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.647349:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:1.0:1789762791.647354:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762791.647361:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175026 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.647378:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175026 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762791.658734:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967587 is committed 00002000:00010000:3.0:1789762791.658748:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 15728640 -> 16777216 00002000:00010000:3.0:1789762791.658753:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 30727 -> 32775 00010000:00010000:3.0:1789762791.658764:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175026 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.658774:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175026 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.658781:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175026 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.658786:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762791.658789:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175026 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.658797:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175026 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.658812:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175026 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.658817:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175026 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.658883:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067328256:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11615us (47222us total) trans 4294967587 rc 0/0 00000100:00100000:3.0:1789762791.658903:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 46 00000100:00100000:3.0:1789762791.658962:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264894219324 time 264894313805 state 0x2 ref 0 00000100:00100000:2.0:1789762791.661552:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.661608:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067328384 00000100:00100000:3.0:1789762791.661616:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067328384 00000100:00100000:3.0:1789762791.661655:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264894219324 time 264894313805 state 0x0 ref 1 00000100:00100000:3.0:1789762791.661667:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264944313805 time 264894313805 state 0x1 ref 1 00000100:00100000:2.0:1789762791.697343:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264944313805 time 264944410036 state 0x0 ref 1 00000100:00100000:2.0:1789762791.697366:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067328384:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762791.697415:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 16777216, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x8007 00010000:00010000:2.0:1789762791.697423:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17502d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.697436:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:2.0:1789762791.697440:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762791.697444:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17502d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.697455:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17502d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762791.708643:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967588 is committed 00002000:00010000:1.0:1789762791.708670:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 16777216 -> 17825792 00002000:00010000:1.0:1789762791.708680:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:1.0:1789762791.708692:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17502d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.708701:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17502d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.708707:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17502d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.708712:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762791.708716:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17502d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.708723:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17502d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.708738:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17502d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.708743:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f17502d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762791.708812:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067328384:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11451us (47256us total) trans 4294967588 rc 0/0 00000100:00100000:1.0:1789762791.708823:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 47 00000100:00100000:1.0:1789762791.708835:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264944313805 time 264944410036 state 0x2 ref 0 00000100:00100000:0.0:1789762791.711705:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.711755:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067328512 00000100:00100000:3.0:1789762791.711765:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067328512 00000100:00100000:3.0:1789762791.711810:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264944313805 time 264944410036 state 0x0 ref 1 00000100:00100000:3.0:1789762791.711825:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264994410036 time 264944410036 state 0x1 ref 1 00000100:00100000:3.0:1789762791.747390:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264994410036 time 264994455982 state 0x0 ref 1 00000100:00100000:3.0:1789762791.747418:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067328512:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762791.747469:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 17825792, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x8807 00010000:00010000:3.0:1789762791.747476:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175034 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.747489:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=264s v=(0 0) 00010000:00010000:3.0:1789762791.747493:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762791.747509:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175034 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.747520:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175034 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762791.758495:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 17825792 -> 18874368 00002000:00010000:2.0:1789762791.758501:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 34823 -> 36871 00000001:00080000:1.0:1789762791.758507:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967589 is committed 00010000:00010000:2.0:1789762791.758511:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175034 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.758522:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175034 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.758529:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175034 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.758541:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762791.758545:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175034 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.758551:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175034 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.758567:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175034 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.758573:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007912bdc3/0xdf5a7ac59f175034 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762791.758635:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067328512:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11227us (46930us total) trans 4294967589 rc 0/0 00000100:00100000:2.0:1789762791.758644:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 48 00000100:00100000:2.0:1789762791.758653:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264994410036 time 264994455982 state 0x2 ref 0 00000100:00100000:0.0:1789762791.760985:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.761015:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067328640 00000100:00100000:3.0:1789762791.761024:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067328640 00000100:00100000:3.0:1789762791.761059:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 264994410036 time 264994455982 state 0x0 ref 1 00000100:00100000:3.0:1789762791.761069:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265044455982 time 264994455982 state 0x1 ref 1 00000100:00100000:1.0:1789762791.797475:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265044455982 time 265044541629 state 0x0 ref 1 00000100:00100000:1.0:1789762791.797498:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067328640:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762791.797546:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 18874368, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x9007 00010000:00010000:1.0:1789762791.797553:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f17503b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.797565:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:1.0:1789762791.797569:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762791.797574:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f17503b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.797584:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f17503b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762791.808288:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:1.0:1789762791.808294:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:1.0:1789762791.808306:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f17503b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.808320:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f17503b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.808327:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f17503b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.808338:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762791.808342:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f17503b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.808349:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f17503b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.808363:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f17503b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.808367:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f17503b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762791.808427:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2397:x1876702067328640:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10937us (47442us total) trans 4294967590 rc 0/0 00000100:00100000:1.0:1789762791.808436:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 49 00000100:00100000:1.0:1789762791.808445:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265044455982 time 265044541629 state 0x2 ref 0 00000001:00080000:1.0:1789762791.808476:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967590 is committed 00000100:00100000:2.0:1789762791.810567:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762791.810635:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067328768 00000100:00100000:1.0:1789762791.810644:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067328768 00000100:00100000:1.0:1789762791.810681:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265044455982 time 265044541629 state 0x0 ref 1 00000100:00100000:1.0:1789762791.810692:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265094541629 time 265044541629 state 0x1 ref 1 00000100:00100000:2.0:1789762791.847592:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265094541629 time 265094654158 state 0x0 ref 1 00000100:00100000:2.0:1789762791.847620:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067328768:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762791.847667:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 19922944, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0x9807 00010000:00010000:2.0:1789762791.847687:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175042 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.847700:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:2.0:1789762791.847704:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762791.847708:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175042 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.847719:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175042 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762791.859050:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967591 is committed 00002000:00010000:1.0:1789762791.859062:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 19922944 -> 20971520 00002000:00010000:1.0:1789762791.859067:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 38919 -> 40969 00010000:00010000:1.0:1789762791.859079:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175042 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.859093:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175042 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.859103:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175042 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.859112:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762791.859117:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175042 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.859126:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175042 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.859144:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175042 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.859150:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175042 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762791.859209:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067328768:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11598us (48641us total) trans 4294967591 rc 0/0 00000100:00100000:1.0:1789762791.859225:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 50 00000100:00100000:1.0:1789762791.859233:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265094541629 time 265094654158 state 0x2 ref 0 00000100:00100000:0.0:1789762791.861481:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762791.861532:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067328896 00000100:00100000:2.0:1789762791.861541:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067328896 00000100:00100000:2.0:1789762791.861576:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265094541629 time 265094654158 state 0x0 ref 1 00000100:00100000:2.0:1789762791.861587:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265144654158 time 265094654158 state 0x1 ref 1 00000100:00100000:2.0:1789762791.879774:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762791.879807:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762791.879825:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.879841:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067329024 00000100:00100000:3.0:1789762791.879848:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067329024 00000100:00100000:2.0:1789762791.879852:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762791.879869:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067329152 00000100:00100000:1.0:1789762791.879871:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067329280 00000100:00100000:0.0:1789762791.879877:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067329152 00000100:00100000:1.0:1789762791.879877:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067329280 00000100:00100000:2.0:1789762791.879883:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762791.879896:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 329 00000100:00100000:3.0:1789762791.879942:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067329024:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762791.879960:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067329536 00000100:00100000:0.0:1789762791.879967:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067329536 00000100:00100000:1.0:1789762791.879968:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 338 00000100:00100000:2.0:1789762791.879969:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762791.879983:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:2398:x1876702067329152:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762791.879992:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067329664 00000100:00100000:0.0:1789762791.879996:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067329664 00000100:00100000:3.0:1789762791.879998:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067329024:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 92us (187us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762791.880007:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 329 00000100:00100000:0.0:1789762791.880026:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 339 00000100:00100000:1.0:1789762791.880031:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:2398:x1876702067329152:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 52us (261us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762791.880036:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2397:x1876702067329280:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762791.880041:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 338 00000100:00100000:3.0:1789762791.880063:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 340 00000100:00100000:1.0:1789762791.880077:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 341 00000100:00100000:0.0:1789762791.880080:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2397:x1876702067329280:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 40us (247us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762791.880080:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067329536:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762791.880091:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067329664:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762791.880101:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 339 00000100:00100000:1.0:1789762791.880136:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067329664:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 50us (173us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762791.880144:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 341 00000100:00100000:2.0:1789762791.880150:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067329408 00000100:00080000:0.0:1789762791.880152:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762791.880156:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067329408 00000100:00080000:0.0:1789762791.880158:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762791.880182:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067329536:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 110us (298us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762791.880189:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 340 00000100:00080000:1.0:1789762791.880194:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762791.880202:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 130 00000100:00100000:2.0:1789762791.880214:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2398:x1876702067329408:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762791.880264:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2398:x1876702067329408:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 52us (413us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762791.880274:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 130 00000100:00100000:2.0:1789762791.897638:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265144654158 time 265144708218 state 0x0 ref 1 00000100:00100000:2.0:1789762791.897653:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067328896:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762791.897693:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 20971520, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0xa009 00010000:00010000:2.0:1789762791.897700:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f175049 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.897712:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:2.0:1789762791.897716:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762791.897721:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f175049 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.897732:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f175049 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762791.908940:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967592 is committed 00002000:00010000:0.0:1789762791.908950:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:0.0:1789762791.908954:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 40969 -> 43017 00010000:00010000:0.0:1789762791.908968:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f175049 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.908983:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f175049 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.908993:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f175049 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.909001:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762791.909007:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f175049 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.909017:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f175049 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.909035:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f175049 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.909041:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f175049 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762791.909119:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067328896:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11466us (47636us total) trans 4294967592 rc 0/0 00000100:00100000:0.0:1789762791.909128:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 51 00000100:00100000:0.0:1789762791.909137:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265144654158 time 265144708218 state 0x2 ref 0 00000100:00100000:3.0:1789762791.911266:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762791.911345:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067329792 00000100:00100000:1.0:1789762791.911353:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067329792 00000100:00100000:1.0:1789762791.911390:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265144654158 time 265144708218 state 0x0 ref 1 00000100:00100000:1.0:1789762791.911401:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265194708218 time 265144708218 state 0x1 ref 1 00000001:00080000:3.0:1789762791.944407:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967377 is committed 00000020:00080000:3.0:1789762791.944422:0:7221:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9c37f7fbe240 refcount 2 committed 0 result 0 batchid 6 00000020:00080000:3.0:1789762791.944427:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37ff763f00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9c37f7fbef00 00000020:00080000:3.0:1789762791.944430:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x2:0x0].6 00000020:00080000:3.0:1789762791.944432:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37ff763600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9c37d927f800 00000020:00080000:3.0:1789762791.944434:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x2:0x0].6 00010000:00010000:2.0:1789762791.944436:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c91 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944447:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c91 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762791.944448:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 9 committed 00000020:00080000:0.0:1789762791.944457:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 4 00010000:00010000:2.0:1789762791.944457:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c91 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762791.944460:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 9 committed 00010000:00010000:2.0:1789762791.944478:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762791.944483:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c91 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1789762791.944488:0:7221:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9c3801da49c0 refcount 2 committed 0 result 0 batchid 5 00010000:00010000:2.0:1789762791.944489:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c91 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1789762791.944498:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37ff807900 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9c3801da4540 00000020:00080000:3.0:1789762791.944504:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x2:0x0].5 00000020:00080000:3.0:1789762791.944507:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37ff807600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9c37d2032000 00010000:00010000:2.0:1789762791.944511:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00000020:00080000:3.0:1789762791.944511:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x2:0x0].5 00010000:00010000:2.0:1789762791.944516:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762791.944519:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c91 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944524:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6000/0xdf5a7ac59f174c91 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944541:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f174d01 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944548:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f174d01 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944555:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f174d01 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1789762791.944563:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 6 committed batchid 4 00010000:00010000:2.0:1789762791.944563:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11000 already unused, calling callback (ffffffffc1f644f0) 00000100:00100000:1.0:1789762791.944564:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00010000:00010000:2.0:1789762791.944570:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f174d01 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944581:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f174d01 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762791.944590:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074652032:0@lo:6:osp-syn-1-1.0 00010000:00010000:2.0:1789762791.944608:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:2.0:1789762791.944611:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789762791.944614:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f174d01 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944618:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b11000/0xdf5a7ac59f174d01 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944630:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7e00/0xdf5a7ac59f174d16 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944635:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7e00/0xdf5a7ac59f174d16 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944640:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7e00/0xdf5a7ac59f174d16 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944645:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762791.944648:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7e00/0xdf5a7ac59f174d16 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944653:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7e00/0xdf5a7ac59f174d16 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762791.944663:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789762791.944666:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x0 (ffff9c37e125a600) 00010000:00010000:2.0:1789762791.944669:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762791.944671:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7e00/0xdf5a7ac59f174d16 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944675:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7e00/0xdf5a7ac59f174d16 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.944699:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074652032 00000100:00100000:3.0:1789762791.944706:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074652032 00010000:00010000:2.0:1789762791.944707:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10800/0xdf5a7ac59f174d55 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944713:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10800/0xdf5a7ac59f174d55 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1789762791.944715:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 4 00010000:00010000:2.0:1789762791.944720:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10800/0xdf5a7ac59f174d55 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762791.944725:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2758700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074652160:0@lo:6:osp-syn-0-1.0 00010000:00010000:2.0:1789762791.944726:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762791.944730:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10800/0xdf5a7ac59f174d55 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944738:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10800/0xdf5a7ac59f174d55 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944744:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x73550ecb (ffff9c37d3bee200) 00010000:00010000:2.0:1789762791.944746:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762791.944749:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10800/0xdf5a7ac59f174d55 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944753:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10800/0xdf5a7ac59f174d55 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.944760:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 126 00000100:00100000:1.0:1789762791.944766:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789762791.944769:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7400/0xdf5a7ac59f174d94 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.944775:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074652032:12345-0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1789762791.944776:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074652160 00010000:00010000:2.0:1789762791.944778:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7400/0xdf5a7ac59f174d94 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762791.944785:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074652160 00010000:00010000:2.0:1789762791.944788:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7400/0xdf5a7ac59f174d94 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944797:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762791.944800:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7400/0xdf5a7ac59f174d94 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1789762791.944802:0:8053:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:7 count 1 00000100:00080000:1.0:1789762791.944803:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789762791.944808:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7400/0xdf5a7ac59f174d94 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789762791.944811:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1789762791.944819:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x2b0e463d (ffff9c37e125aa00) 00010000:00010000:2.0:1789762791.944823:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762791.944824:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00010000:00010000:2.0:1789762791.944826:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7400/0xdf5a7ac59f174d94 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762791.944831:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x7:0x0] initial LVB size: 8388608, mtime: 0x6aad9ce6, atime: 0x0, ctime: 0x6aad9ce6, blocks: 0x1 00010000:00010000:2.0:1789762791.944833:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa7400/0xdf5a7ac59f174d94 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762791.944841:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074652160:12345-0@lo:6:osp-syn-0-1.0 00010000:00010000:3.0:1789762791.944844:0:8053:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f175050 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944849:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6400/0xdf5a7ac59f174e7b lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1789762791.944861:0:9038:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:7 count 1 00010000:00010000:2.0:1789762791.944869:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6400/0xdf5a7ac59f174e7b lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.944872:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=265s v=(0 0) 00010000:00010000:2.0:1789762791.944878:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6400/0xdf5a7ac59f174e7b lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.944879:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x7:0x0].0x0 since=0s v=(0 0) 00002000:00010000:0.0:1789762791.944885:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x7:0x0] initial LVB size: 8388608, mtime: 0x6aad9ce6, atime: 0x0, ctime: 0x6aad9ce6, blocks: 0x4007 00010000:00010000:3.0:1789762791.944890:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f175050 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944891:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762791.944896:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6400/0xdf5a7ac59f174e7b lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.944899:0:9038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f175057 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.944956:0:8053:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f175050 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944961:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6400/0xdf5a7ac59f174e7b lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.944965:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=265s v=(0 0) 00010000:00010000:0.0:1789762791.944977:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762791.944989:0:8053:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f175050 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.944991:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f175057 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.944995:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x2b0e463d (ffff9c37e125aa00) 00010000:00010000:2.0:1789762791.945002:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762791.945007:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6400/0xdf5a7ac59f174e7b lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.945015:0:8053:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f175050 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.945025:0:9038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f175057 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.945026:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d1fa6400/0xdf5a7ac59f174e7b lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.945044:0:8053:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762791.945054:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e97 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.945055:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762791.945064:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f175050 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.945069:0:9038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f175057 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.945070:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e97 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.945082:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762791.945087:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e97 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.945091:0:9038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f175057 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.945091:0:6408:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f175050 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.945094:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762791.945099:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e97 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.945115:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e97 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.945116:0:6408:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f175050 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762791.945118:0:9038:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762791.945129:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xb:0x0].0x73550ecb (ffff9c37d3bee200) 00010000:00010000:2.0:1789762791.945132:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762791.945135:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e97 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.945143:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f175050 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762791.945143:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37e1b10400/0xdf5a7ac59f174e97 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xb:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.945167:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052b012fc/0xdf5a7ac59f175050 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.945188:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762791.945192:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f175057 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.945212:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762791.945217:0:6408:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f175057 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.945226:0:6408:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f175057 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.945240:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f175057 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.945248:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045e48c30/0xdf5a7ac59f175057 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762791.945291:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762791.945301:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762791.945304:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762791.945307:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762791.945732:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074652032:12345-0@lo:6:osp-syn-1-1.0 Request processed in 962us (1074us total) trans 4294967362 rc 0/0 00000100:00100000:3.0:1789762791.945739:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 126 00000004:00080000:1.0:1789762791.945769:0:3426:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37f7f5aa00/2, rc 0, transno 4294967362 00000100:00100000:0.0:1789762791.945778:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074652160:12345-0@lo:6:osp-syn-0-1.0 Request processed in 943us (1016us total) trans 4294967593 rc 0/0 00000100:00100000:0.0:1789762791.945786:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00000100:00100000:1.0:1789762791.945787:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074652032:0@lo:6:osp-syn-1-1.0 00000004:00080000:1.0:1789762791.945800:0:3426:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37d2758700/2, rc 0, transno 4294967593 00000100:00100000:1.0:1789762791.945813:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2758700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074652160:0@lo:6:osp-syn-0-1.0 00000100:00100000:3.0:1789762791.947689:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265194708218 time 265194760363 state 0x0 ref 1 00000100:00100000:3.0:1789762791.947700:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067329792:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762791.947722:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 22020096, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0xa809 00010000:00010000:3.0:1789762791.947727:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17505e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.947734:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:3.0:1789762791.947737:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762791.947740:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17505e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.947747:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17505e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762791.963146:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967593 is committed 00002000:00010000:3.0:1789762791.963161:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 22020096 -> 23068672 00000001:00080000:0.0:1789762791.963163:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967594 is committed 00002000:00010000:3.0:1789762791.963167:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 43017 -> 45065 00010000:00010000:3.0:1789762791.963179:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17505e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.963190:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17505e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.963196:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17505e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.963201:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762791.963205:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17505e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.963211:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17505e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.963226:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17505e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.963231:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17505e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762791.963303:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067329792:12345-192.168.203.37@tcp:4:dd.500 Request processed in 15602us (52035us total) trans 4294967594 rc 0/0 00000100:00100000:3.0:1789762791.963318:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 52 00000100:00100000:3.0:1789762791.963329:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265194708218 time 265194760363 state 0x2 ref 0 00000100:00100000:2.0:1789762791.965839:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762791.965884:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067329920 00000100:00100000:1.0:1789762791.965893:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067329920 00000100:00100000:1.0:1789762791.965971:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265194708218 time 265194760363 state 0x0 ref 1 00000100:00100000:1.0:1789762791.966002:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265244760363 time 265194760363 state 0x1 ref 1 00000100:00100000:3.0:1789762791.997798:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265244760363 time 265244864163 state 0x0 ref 1 00000100:00100000:3.0:1789762791.997821:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067329920:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762791.997868:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 23068672, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0xb009 00010000:00010000:3.0:1789762791.997875:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175065 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.997888:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:3.0:1789762791.997892:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762791.997897:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175065 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762791.997943:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175065 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762792.009304:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967595 is committed 00002000:00010000:3.0:1789762792.009323:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:3.0:1789762792.009328:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 45065 -> 47113 00010000:00010000:3.0:1789762792.009339:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175065 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.009348:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175065 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.009364:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175065 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.009370:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.009373:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175065 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.009379:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175065 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.009394:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175065 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.009400:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175065 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.009462:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067329920:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11648us (43621us total) trans 4294967595 rc 0/0 00000100:00100000:3.0:1789762792.009470:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 53 00000100:00100000:3.0:1789762792.009480:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265244760363 time 265244864163 state 0x2 ref 0 00000100:00100000:2.0:1789762792.011846:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762792.011942:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067330048 00000100:00100000:1.0:1789762792.011950:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067330048 00000100:00100000:1.0:1789762792.011998:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265244760363 time 265244864163 state 0x0 ref 1 00000100:00100000:1.0:1789762792.012009:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265294864163 time 265244864163 state 0x1 ref 1 00000100:00100000:3.0:1789762792.047949:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265294864163 time 265294977642 state 0x0 ref 1 00000100:00100000:3.0:1789762792.047982:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067330048:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762792.048053:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 24117248, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0xb809 00010000:00010000:3.0:1789762792.048063:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17506c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.048077:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:3.0:1789762792.048081:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762792.048085:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17506c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.048097:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17506c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762792.060286:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967596 is committed 00002000:00010000:1.0:1789762792.060330:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 24117248 -> 25165824 00002000:00010000:1.0:1789762792.060333:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 47113 -> 49161 00010000:00010000:1.0:1789762792.060344:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17506c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.060355:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17506c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.060361:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17506c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.060366:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762792.060370:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17506c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.060376:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17506c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.060404:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17506c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.060414:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f17506c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762792.060499:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067330048:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12527us (48649us total) trans 4294967596 rc 0/0 00000100:00100000:1.0:1789762792.060511:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 54 00000100:00100000:1.0:1789762792.060521:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265294864163 time 265294977642 state 0x2 ref 0 00000100:00100000:1.0:1789762792.063279:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.063327:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067330176 00000100:00100000:3.0:1789762792.063337:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067330176 00000100:00100000:3.0:1789762792.063382:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265294864163 time 265294977642 state 0x0 ref 1 00000100:00100000:3.0:1789762792.063398:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265344977642 time 265294977642 state 0x1 ref 1 00010000:00010000:1.0:1789762792.064002:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762792.098013:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265344977642 time 265345074108 state 0x0 ref 1 00000100:00100000:2.0:1789762792.098043:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067330176:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762792.098096:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 25165824, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0xc009 00010000:00010000:2.0:1789762792.098102:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175073 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762792.098116:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:2.0:1789762792.098120:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762792.098124:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175073 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762792.098135:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175073 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762792.110086:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967597 is committed 00002000:00010000:3.0:1789762792.110112:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:3.0:1789762792.110119:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 49161 -> 51209 00010000:00010000:3.0:1789762792.110132:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175073 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.110146:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175073 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.110171:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175073 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.110179:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.110185:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175073 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.110195:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175073 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.110217:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175073 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.110226:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175073 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.110301:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067330176:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12268us (47020us total) trans 4294967597 rc 0/0 00000100:00100000:3.0:1789762792.110312:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 55 00000100:00100000:3.0:1789762792.110325:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265344977642 time 265345074108 state 0x2 ref 0 00000100:00100000:2.0:1789762792.112691:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.112733:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067330304 00000100:00100000:3.0:1789762792.112741:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067330304 00000100:00100000:3.0:1789762792.112788:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265344977642 time 265345074108 state 0x0 ref 1 00000100:00100000:3.0:1789762792.112814:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265395074108 time 265345074108 state 0x1 ref 1 00000100:00100000:3.0:1789762792.148054:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265395074108 time 265395122109 state 0x0 ref 1 00000100:00100000:3.0:1789762792.148075:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067330304:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762792.148121:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 26214400, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0xc809 00010000:00010000:3.0:1789762792.148127:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f17507a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.148140:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:3.0:1789762792.148143:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762792.148148:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f17507a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.148159:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f17507a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762792.160295:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967598 is committed 00002000:00010000:1.0:1789762792.160296:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 26214400 -> 27262976 00002000:00010000:1.0:1789762792.160303:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 51209 -> 53257 00010000:00010000:1.0:1789762792.160314:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f17507a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.160323:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f17507a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.160331:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f17507a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.160336:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762792.160345:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f17507a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.160352:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f17507a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.160367:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f17507a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.160372:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ceb8a91/0xdf5a7ac59f17507a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762792.160434:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067330304:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12364us (47741us total) trans 4294967598 rc 0/0 00000100:00100000:1.0:1789762792.160442:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 56 00000100:00100000:1.0:1789762792.160452:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265395074108 time 265395122109 state 0x2 ref 0 00000100:00100000:2.0:1789762792.162615:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762792.162651:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067330432 00000100:00100000:1.0:1789762792.162659:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067330432 00000100:00100000:1.0:1789762792.162696:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265395074108 time 265395122109 state 0x0 ref 1 00000100:00100000:1.0:1789762792.162706:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265445122109 time 265395122109 state 0x1 ref 1 00000100:00100000:1.0:1789762792.198104:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265445122109 time 265445176478 state 0x0 ref 1 00000100:00100000:1.0:1789762792.198119:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067330432:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762792.198160:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 27262976, mtime: 0x6aad9ce7, atime: 0x0, ctime: 0x6aad9ce7, blocks: 0xd009 00010000:00010000:1.0:1789762792.198166:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175081 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.198178:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:1.0:1789762792.198182:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762792.198186:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175081 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.198196:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175081 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762792.210639:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967599 is committed 00002000:00010000:1.0:1789762792.210653:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:1.0:1789762792.210658:0:8057:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb mtime from disk: 1789762791 -> 1789762792 00002000:00010000:1.0:1789762792.210661:0:8057:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb ctime from disk: 1789762791 -> 1789762792 00002000:00010000:1.0:1789762792.210663:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 53257 -> 55305 00010000:00010000:1.0:1789762792.210672:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175081 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.210682:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175081 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.210688:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175081 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.210693:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762792.210703:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175081 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.210710:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175081 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.210725:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175081 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.210731:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175081 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762792.210789:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067330432:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12671us (48173us total) trans 4294967599 rc 0/0 00000100:00100000:1.0:1789762792.210797:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 57 00000100:00100000:1.0:1789762792.210806:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265445122109 time 265445176478 state 0x2 ref 0 00000100:00100000:2.0:1789762792.213150:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.213190:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067330560 00000100:00100000:3.0:1789762792.213197:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067330560 00000100:00100000:3.0:1789762792.213233:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265445122109 time 265445176478 state 0x0 ref 1 00000100:00100000:3.0:1789762792.213243:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265495176478 time 265445176478 state 0x1 ref 1 00000100:00100000:1.0:1789762792.248202:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265495176478 time 265495268404 state 0x0 ref 1 00000100:00100000:1.0:1789762792.248227:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067330560:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762792.248275:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 28311552, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0xd809 00010000:00010000:1.0:1789762792.248282:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175088 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.248294:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:1.0:1789762792.248309:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762792.248314:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175088 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.248325:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175088 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762792.259294:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967600 is committed 00002000:00010000:3.0:1789762792.259344:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 28311552 -> 29360128 00002000:00010000:3.0:1789762792.259349:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 55305 -> 57353 00010000:00010000:3.0:1789762792.259359:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175088 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.259368:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175088 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.259375:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175088 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.259380:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.259383:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175088 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.259390:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175088 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.259403:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175088 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.259408:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175088 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.259468:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067330560:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11249us (46317us total) trans 4294967600 rc 0/0 00000100:00100000:3.0:1789762792.259486:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 58 00000100:00100000:3.0:1789762792.259494:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265495176478 time 265495268404 state 0x2 ref 0 00000100:00100000:0.0:1789762792.261669:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762792.261719:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067330688 00000100:00100000:1.0:1789762792.261726:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067330688 00000100:00100000:1.0:1789762792.261764:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265495176478 time 265495268404 state 0x0 ref 1 00000100:00100000:1.0:1789762792.261774:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265545268404 time 265495268404 state 0x1 ref 1 00000100:00100000:3.0:1789762792.298280:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265545268404 time 265545349315 state 0x0 ref 1 00000100:00100000:3.0:1789762792.298300:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79ad850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067330688:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762792.298347:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 29360128, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0xe009 00010000:00010000:3.0:1789762792.298356:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17508f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.298370:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:3.0:1789762792.298374:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762792.298379:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17508f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.298389:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17508f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762792.309303:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967601 is committed 00002000:00010000:3.0:1789762792.309332:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 29360128 -> 30408704 00002000:00010000:3.0:1789762792.309339:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 57353 -> 59401 00010000:00010000:3.0:1789762792.309351:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17508f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.309366:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17508f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.309386:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17508f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.309391:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.309395:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17508f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.309401:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17508f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.309415:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17508f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.309420:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17508f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.309496:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79ad850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067330688:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11200us (47826us total) trans 4294967601 rc 0/0 00000100:00100000:3.0:1789762792.309506:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 59 00000100:00100000:3.0:1789762792.309515:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265545268404 time 265545349315 state 0x2 ref 0 00000100:00100000:2.0:1789762792.312129:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762792.312178:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067330816 00000100:00100000:1.0:1789762792.312187:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067330816 00000100:00100000:1.0:1789762792.312225:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265545268404 time 265545349315 state 0x0 ref 1 00000100:00100000:1.0:1789762792.312236:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265595349315 time 265545349315 state 0x1 ref 1 00000100:00100000:1.0:1789762792.348385:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265595349315 time 265595452815 state 0x0 ref 1 00000100:00100000:1.0:1789762792.348406:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff617480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067330816:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762792.348451:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 30408704, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0xe809 00010000:00010000:1.0:1789762792.348458:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175096 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.348483:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:1.0:1789762792.348486:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762792.348491:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175096 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.348503:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175096 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762792.360609:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967602 is committed 00002000:00010000:3.0:1789762792.360628:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 30408704 -> 31457280 00002000:00010000:3.0:1789762792.360633:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 59401 -> 61449 00010000:00010000:3.0:1789762792.360645:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175096 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.360665:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175096 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.360671:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175096 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.360676:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.360679:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175096 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.360686:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175096 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.360701:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175096 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.360705:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175096 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.360770:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff617480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067330816:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12369us (48639us total) trans 4294967602 rc 0/0 00000100:00100000:3.0:1789762792.360780:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 60 00000100:00100000:3.0:1789762792.360791:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265595349315 time 265595452815 state 0x2 ref 0 00000100:00100000:2.0:1789762792.363065:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.363104:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067330944 00000100:00100000:3.0:1789762792.363120:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067330944 00000100:00100000:3.0:1789762792.363166:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265595349315 time 265595452815 state 0x0 ref 1 00000100:00100000:3.0:1789762792.363182:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265645452815 time 265595452815 state 0x1 ref 1 00000100:00100000:3.0:1789762792.398448:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265645452815 time 265645515438 state 0x0 ref 1 00000100:00100000:3.0:1789762792.398470:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067330944:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762792.398516:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 31457280, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0xf009 00010000:00010000:3.0:1789762792.398523:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17509d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.398535:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:3.0:1789762792.398539:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762792.398544:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17509d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.398554:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17509d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762792.410149:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967603 is committed 00002000:00010000:3.0:1789762792.410159:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 31457280 -> 32505856 00002000:00010000:3.0:1789762792.410164:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 61449 -> 63497 00010000:00010000:3.0:1789762792.410173:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17509d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.410183:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17509d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.410196:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17509d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.410201:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.410205:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17509d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.410211:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17509d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.410225:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17509d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.410231:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17509d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.410289:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067330944:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11826us (47223us total) trans 4294967603 rc 0/0 00000100:00100000:3.0:1789762792.410297:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 61 00000100:00100000:3.0:1789762792.410305:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265645452815 time 265645515438 state 0x2 ref 0 00000100:00100000:0.0:1789762792.412726:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.412761:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067331072 00000100:00100000:3.0:1789762792.412770:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067331072 00000100:00100000:3.0:1789762792.412806:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265645452815 time 265645515438 state 0x0 ref 1 00000100:00100000:3.0:1789762792.412826:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265695515438 time 265645515438 state 0x1 ref 1 00000100:00100000:3.0:1789762792.448500:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265695515438 time 265695569567 state 0x0 ref 1 00000100:00100000:3.0:1789762792.448519:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067331072:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762792.448564:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 32505856, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0xf809 00010000:00010000:3.0:1789762792.448573:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.448589:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:3.0:1789762792.448593:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762792.448598:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.448609:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762792.460491:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967604 is committed 00002000:00010000:3.0:1789762792.460534:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 32505856 -> 33554432 00002000:00010000:3.0:1789762792.460538:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 63497 -> 65545 00010000:00010000:3.0:1789762792.460549:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.460558:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.460575:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.460580:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.460583:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.460590:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.460604:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.460610:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.460672:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067331072:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12157us (47946us total) trans 4294967604 rc 0/0 00000100:00100000:3.0:1789762792.460681:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 62 00000100:00100000:3.0:1789762792.460690:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265695515438 time 265695569567 state 0x2 ref 0 00000100:00100000:0.0:1789762792.462847:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.462882:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067331200 00000100:00100000:3.0:1789762792.462890:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067331200 00000100:00100000:3.0:1789762792.462953:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265695515438 time 265695569567 state 0x0 ref 1 00000100:00100000:3.0:1789762792.462974:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265745569567 time 265695569567 state 0x1 ref 1 00000100:00100000:1.0:1789762792.498602:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265745569567 time 265745667143 state 0x0 ref 1 00000100:00100000:1.0:1789762792.498625:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067331200:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762792.498668:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 33554432, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x10009 00010000:00010000:1.0:1789762792.498677:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.498692:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:1.0:1789762792.498696:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762792.498702:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.498716:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762792.510795:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967605 is committed 00002000:00010000:3.0:1789762792.510813:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:3.0:1789762792.510819:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 65545 -> 67593 00010000:00010000:3.0:1789762792.510830:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.510840:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.510847:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.510853:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.510856:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.510862:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.510876:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.510881:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.510996:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067331200:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12375us (48145us total) trans 4294967605 rc 0/0 00000100:00100000:3.0:1789762792.511005:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 63 00000100:00100000:3.0:1789762792.511014:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265745569567 time 265745667143 state 0x2 ref 0 00000100:00100000:2.0:1789762792.513621:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.513655:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067331328 00000100:00100000:3.0:1789762792.513662:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067331328 00000100:00100000:3.0:1789762792.513701:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265745569567 time 265745667143 state 0x0 ref 1 00000100:00100000:3.0:1789762792.513712:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265795667143 time 265745667143 state 0x1 ref 1 00000100:00100000:1.0:1789762792.548717:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265795667143 time 265795783212 state 0x0 ref 1 00000100:00100000:1.0:1789762792.548738:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067331328:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762792.548785:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 34603008, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x10809 00010000:00010000:1.0:1789762792.548792:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.548805:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:1.0:1789762792.548809:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762792.548813:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.548824:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762792.560171:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967606 is committed 00002000:00010000:1.0:1789762792.560220:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:1.0:1789762792.560225:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 67593 -> 69641 00010000:00010000:1.0:1789762792.560235:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.560244:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.560250:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.560266:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762792.560270:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.560276:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.560290:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.560295:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762792.560353:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067331328:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11621us (46731us total) trans 4294967606 rc 0/0 00000100:00100000:1.0:1789762792.560361:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 64 00000100:00100000:1.0:1789762792.560371:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265795667143 time 265795783212 state 0x2 ref 0 00000100:00100000:2.0:1789762792.562510:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.562550:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067331456 00000100:00100000:3.0:1789762792.562558:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067331456 00000100:00100000:3.0:1789762792.562603:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265795667143 time 265795783212 state 0x0 ref 1 00000100:00100000:3.0:1789762792.562618:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265845783212 time 265795783212 state 0x1 ref 1 00000100:00100000:1.0:1789762792.598801:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265845783212 time 265845865590 state 0x0 ref 1 00000100:00100000:1.0:1789762792.598824:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067331456:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762792.598870:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 35651584, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x11009 00010000:00010000:1.0:1789762792.598878:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.598943:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:1.0:1789762792.598950:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762792.598958:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.598974:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762792.609993:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967607 is committed 00002000:00010000:3.0:1789762792.610004:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 35651584 -> 36700160 00002000:00010000:3.0:1789762792.610009:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 69641 -> 71689 00010000:00010000:3.0:1789762792.610019:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.610029:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.610046:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.610051:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.610055:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.610061:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.610075:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.610081:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1750b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.610149:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067331456:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11328us (47635us total) trans 4294967607 rc 0/0 00000100:00100000:3.0:1789762792.610160:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 65 00000100:00100000:3.0:1789762792.610171:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265845783212 time 265845865590 state 0x2 ref 0 00000100:00100000:2.0:1789762792.612314:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.612349:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067331584 00000100:00100000:3.0:1789762792.612357:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067331584 00000100:00100000:3.0:1789762792.612393:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265845783212 time 265845865590 state 0x0 ref 1 00000100:00100000:3.0:1789762792.612413:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265895865590 time 265845865590 state 0x1 ref 1 00000100:00100000:1.0:1789762792.648952:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265895865590 time 265895992954 state 0x0 ref 1 00000100:00100000:1.0:1789762792.648976:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067331584:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762792.649021:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 36700160, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x11809 00010000:00010000:1.0:1789762792.649028:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.649040:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:1.0:1789762792.649044:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762792.649049:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.649060:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762792.660157:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967608 is committed 00002000:00010000:1.0:1789762792.660224:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 36700160 -> 37748736 00002000:00010000:1.0:1789762792.660231:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 71689 -> 73737 00010000:00010000:1.0:1789762792.660248:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.660258:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.660264:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.660269:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762792.660273:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.660279:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.660293:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.660297:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762792.660354:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067331584:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11386us (48038us total) trans 4294967608 rc 0/0 00000100:00100000:1.0:1789762792.660363:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 66 00000100:00100000:1.0:1789762792.660372:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265895865590 time 265895992954 state 0x2 ref 0 00000100:00100000:0.0:1789762792.662527:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762792.662568:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067331712 00000100:00100000:1.0:1789762792.662579:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067331712 00000100:00100000:1.0:1789762792.662624:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265895865590 time 265895992954 state 0x0 ref 1 00000100:00100000:1.0:1789762792.662638:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265945992954 time 265895992954 state 0x1 ref 1 00000100:00100000:1.0:1789762792.698968:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265945992954 time 265946039551 state 0x0 ref 1 00000100:00100000:1.0:1789762792.698985:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067331712:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762792.699023:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 37748736, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x12009 00010000:00010000:1.0:1789762792.699032:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.699047:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:1.0:1789762792.699052:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762792.699059:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.699074:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762792.709843:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967609 is committed 00002000:00010000:1.0:1789762792.709876:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:1.0:1789762792.709882:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 73737 -> 75785 00010000:00010000:1.0:1789762792.709896:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.709951:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.709961:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.709968:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762792.709973:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.709983:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.710003:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.710011:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762792.710092:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067331712:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11107us (47561us total) trans 4294967609 rc 0/0 00000100:00100000:1.0:1789762792.710104:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 67 00000100:00100000:1.0:1789762792.710116:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265945992954 time 265946039551 state 0x2 ref 0 00000100:00100000:2.0:1789762792.712825:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.712867:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067331840 00000100:00100000:3.0:1789762792.712878:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067331840 00000100:00100000:3.0:1789762792.712951:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265945992954 time 265946039551 state 0x0 ref 1 00000100:00100000:3.0:1789762792.712963:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265996039551 time 265946039551 state 0x1 ref 1 00000100:00100000:1.0:1789762792.749067:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265996039551 time 265996134029 state 0x0 ref 1 00000100:00100000:1.0:1789762792.749098:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067331840:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762792.749142:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 38797312, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x12809 00010000:00010000:1.0:1789762792.749149:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.749162:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=265s v=(0 0) 00010000:00010000:1.0:1789762792.749166:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762792.749171:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.749183:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762792.760055:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967610 is committed 00002000:00010000:3.0:1789762792.760062:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 38797312 -> 39845888 00002000:00010000:3.0:1789762792.760068:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 75785 -> 77833 00010000:00010000:3.0:1789762792.760079:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.760090:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.760096:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.760102:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.760105:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.760112:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.760136:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.760142:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.760203:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067331840:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11108us (47378us total) trans 4294967610 rc 0/0 00000100:00100000:3.0:1789762792.760211:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 68 00000100:00100000:3.0:1789762792.760221:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265996039551 time 265996134029 state 0x2 ref 0 00000100:00100000:2.0:1789762792.762711:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762792.762785:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067331968 00000100:00100000:1.0:1789762792.762794:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067331968 00000100:00100000:1.0:1789762792.762830:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 265996039551 time 265996134029 state 0x0 ref 1 00000100:00100000:1.0:1789762792.762841:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266046134029 time 265996134029 state 0x1 ref 1 00000100:00100000:3.0:1789762792.799144:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266046134029 time 266046210428 state 0x0 ref 1 00000100:00100000:3.0:1789762792.799167:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067331968:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762792.799214:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 39845888, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x13009 00010000:00010000:3.0:1789762792.799221:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.799235:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:3.0:1789762792.799238:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762792.799243:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.799254:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762792.810095:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967611 is committed 00002000:00010000:3.0:1789762792.810155:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 39845888 -> 40894464 00002000:00010000:3.0:1789762792.810161:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 77833 -> 79881 00010000:00010000:3.0:1789762792.810170:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.810179:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.810186:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.810191:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.810194:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.810201:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.810214:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.810219:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f1750d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.810281:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067331968:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11120us (47568us total) trans 4294967611 rc 0/0 00000100:00100000:3.0:1789762792.810295:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 69 00000100:00100000:3.0:1789762792.810305:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266046134029 time 266046210428 state 0x2 ref 0 00000100:00100000:1.0:1789762792.812536:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.812572:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067332096 00000100:00100000:3.0:1789762792.812581:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067332096 00000100:00100000:3.0:1789762792.812617:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266046134029 time 266046210428 state 0x0 ref 1 00000100:00100000:3.0:1789762792.812628:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266096210428 time 266046210428 state 0x1 ref 1 00000100:00100000:1.0:1789762792.849260:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266096210428 time 266096328979 state 0x0 ref 1 00000100:00100000:1.0:1789762792.849281:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067332096:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762792.849331:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 40894464, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x13809 00010000:00010000:1.0:1789762792.849339:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.849351:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762792.849354:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762792.849359:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.849369:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762792.860286:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967612 is committed 00002000:00010000:3.0:1789762792.860343:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 40894464 -> 41943040 00002000:00010000:3.0:1789762792.860348:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 79881 -> 81929 00010000:00010000:3.0:1789762792.860360:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.860374:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.860381:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.860386:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.860390:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.860396:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.860418:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.860424:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.860489:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067332096:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11211us (47950us total) trans 4294967612 rc 0/0 00000100:00100000:3.0:1789762792.860499:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 70 00000100:00100000:3.0:1789762792.860510:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266096210428 time 266096328979 state 0x2 ref 0 00000100:00100000:0.0:1789762792.863064:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.863098:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067332224 00000100:00100000:3.0:1789762792.863106:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067332224 00000100:00100000:3.0:1789762792.863145:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266096210428 time 266096328979 state 0x0 ref 1 00000100:00100000:3.0:1789762792.863155:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266146328979 time 266096328979 state 0x1 ref 1 00000100:00100000:2.0:1789762792.899409:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266146328979 time 266146475967 state 0x0 ref 1 00000100:00100000:2.0:1789762792.899433:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067332224:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762792.899480:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 41943040, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x14009 00010000:00010000:2.0:1789762792.899488:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f1750e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762792.899501:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:2.0:1789762792.899505:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762792.899509:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f1750e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762792.899520:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f1750e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762792.911963:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967613 is committed 00002000:00010000:3.0:1789762792.911978:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 41943040 -> 42991616 00002000:00010000:3.0:1789762792.911985:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 81929 -> 83977 00010000:00010000:3.0:1789762792.911999:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f1750e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.912013:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f1750e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.912037:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f1750e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.912047:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.912053:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f1750e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.912064:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f1750e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.912084:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f1750e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.912093:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f1750e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.912174:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067332224:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12747us (49107us total) trans 4294967613 rc 0/0 00000100:00100000:3.0:1789762792.912186:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 71 00000100:00100000:3.0:1789762792.912198:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266146328979 time 266146475967 state 0x2 ref 0 00000100:00100000:1.0:1789762792.914711:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.914746:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067332352 00000100:00100000:3.0:1789762792.914754:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067332352 00000100:00100000:3.0:1789762792.914795:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266146328979 time 266146475967 state 0x0 ref 1 00000100:00100000:3.0:1789762792.914815:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266196475967 time 266146475967 state 0x1 ref 1 00000100:00100000:1.0:1789762792.949512:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266196475967 time 266196574664 state 0x0 ref 1 00000100:00100000:1.0:1789762792.949539:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067332352:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762792.949595:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 42991616, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x14809 00010000:00010000:1.0:1789762792.949605:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.949622:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762792.949628:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762792.949635:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.949670:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789762792.960060:0:9018:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 266207139893ns, next at 267207136663ns, rc = 0 00000100:00100000:1.0:1789762792.960093:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ca6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074652544:0@lo:400:kworker.0 00000004:00080000:2.0:1789762792.960093:0:9022:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 266207173302ns, next at 267207169995ns, rc = 0 00000100:00100000:1.0:1789762792.960154:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762792.960215:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ca7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074653056:0@lo:400:kworker.0 00000100:00100000:1.0:1789762792.960248:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762792.960291:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074653568:0@lo:400:kworker.0 00000100:00100000:1.0:1789762792.960309:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762792.960331:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074654080:0@lo:400:kworker.0 00000100:00100000:2.0:1789762792.960346:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074652544 00000100:00100000:2.0:1789762792.960353:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074652544 00000100:00100000:1.0:1789762792.960358:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762792.960399:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:2.0:1789762792.960406:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074653056 00000100:00100000:1.0:1789762792.960407:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074652672:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762792.960411:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074653056 00000100:00100000:1.0:1789762792.960434:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762792.960436:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074653568 00000100:00100000:2.0:1789762792.960440:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074653568 00000100:00100000:0.0:1789762792.960454:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 342 00000100:00100000:1.0:1789762792.960465:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ca7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074653312:0@lo:400:kworker.0 00000100:00100000:0.0:1789762792.960467:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074652544:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762792.960477:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 343 00000100:00100000:2.0:1789762792.960489:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074653056:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762792.960497:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762792.960522:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074653696:0@lo:400:kworker.0 00000100:00100000:0.0:1789762792.960528:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074652544:12345-0@lo:400:kworker.0 Request processed in 65us (377us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762792.960537:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 342 00000100:00100000:0.0:1789762792.960555:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:3.0:1789762792.960562:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074652288:0@lo:400:kworker.0 00000100:00100000:0.0:1789762792.960565:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074653568:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762792.960565:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074653056:12345-0@lo:400:kworker.0 Request processed in 75us (312us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762792.960568:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762792.960583:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 343 00000100:00100000:1.0:1789762792.960589:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074654208:0@lo:400:kworker.0 00000100:00100000:2.0:1789762792.960601:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074653696 00000100:00100000:2.0:1789762792.960605:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074653696 00000100:00100000:0.0:1789762792.960624:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074653568:12345-0@lo:400:kworker.0 Request processed in 61us (312us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762792.960629:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 345 00000100:00100000:1.0:1789762792.960638:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762792.960642:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762792.960643:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074653696:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762792.960650:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:1.0:1789762792.960680:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074653568:0@lo:400:kworker.0 00000100:00100000:2.0:1789762792.960696:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074653696:12345-0@lo:400:kworker.0 Request processed in 56us (134us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762792.960702:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074652800:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762792.960704:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 345 00000100:00100000:1.0:1789762792.960709:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ca6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074652544:0@lo:400:kworker.0 00000100:00100000:1.0:1789762792.960722:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ca7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074653056:0@lo:400:kworker.0 00000100:00100000:3.0:1789762792.960747:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762792.960748:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074653696:0@lo:400:kworker.0 00000100:00100000:0.0:1789762792.960775:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074652672 00000100:00100000:3.0:1789762792.960780:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074653184:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762792.960781:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074652672 00000100:00100000:1.0:1789762792.960783:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074654080 00000100:00100000:1.0:1789762792.960789:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074654080 00000100:00100000:0.0:1789762792.960816:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074652800 00000100:00100000:0.0:1789762792.960822:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074652800 00000100:00100000:3.0:1789762792.960824:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762792.960828:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074654208 00000100:00100000:1.0:1789762792.960833:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074654208 00000100:00100000:2.0:1789762792.960847:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074653184 00000100:00100000:2.0:1789762792.960853:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074653184 00000100:00100000:3.0:1789762792.960854:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074653824:0@lo:400:kworker.0 00000100:00100000:0.0:1789762792.960859:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:1.0:1789762792.960864:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00100000:0.0:1789762792.960876:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074652672:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762792.960878:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+7:3426:x1876702074654080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762792.960884:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 143 00000100:00100000:3.0:1789762792.960892:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762792.960914:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3424:x1876702074652800:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762792.960915:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 144637952 pending 1703936 free 3824156672 avail 3797942272 00002000:00000024:2.0:1789762792.960951:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 144637952 pending 1703936 free 3824156672 avail 3797942272 00000100:00100000:3.0:1789762792.960966:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074654336:0@lo:400:kworker.0 00000100:00100000:1.0:1789762792.960972:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+7:3426:x1876702074654080:12345-0@lo:400:kworker.0 Request processed in 96us (612us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762792.960984:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00100000:2.0:1789762792.961002:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3424:x1876702074652800:12345-0@lo:13:osp-pre-0-0.0 Request processed in 105us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762792.961009:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 132 00000100:00100000:0.0:1789762792.961015:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074652672:12345-0@lo:13:osp-pre-0-1.0 Request processed in 142us (579us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762792.961019:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762792.961020:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 143 00000100:00100000:1.0:1789762792.961027:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074654208:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762792.961032:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00100000:2.0:1789762792.961049:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 144 00000100:00100000:2.0:1789762792.961061:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074653184:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762792.961079:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074654208:12345-0@lo:400:kworker.0 Request processed in 56us (446us total) trans 0 rc 0/0 00002000:00000024:2.0:1789762792.961082:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 96927744 pending 25559040 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789762792.961086:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074652800:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762792.961087:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 132 00000100:00100000:3.0:1789762792.961106:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:1.0:1789762792.961107:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074654336 00000100:00100000:1.0:1789762792.961112:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074654336 00000100:00100000:3.0:1789762792.961120:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074652416:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762792.961128:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074653184:12345-0@lo:13:osp-pre-1-1.0 Request processed in 70us (307us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762792.961137:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 133 00000100:00100000:2.0:1789762792.961137:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 144 00000100:00100000:1.0:1789762792.961150:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074654336:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762792.961156:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762792.961181:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074652416 00000100:00100000:3.0:1789762792.961182:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ca6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074652928:0@lo:400:kworker.0 00000100:00100000:0.0:1789762792.961186:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074652416 00000100:00100000:1.0:1789762792.961191:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074654336:12345-0@lo:400:kworker.0 Request processed in 43us (178us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762792.961197:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 133 00000100:00100000:3.0:1789762792.961212:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762792.961222:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 145 00000100:00100000:1.0:1789762792.961223:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074653824 00000100:00100000:0.0:1789762792.961231:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074652416:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762792.961236:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074653824 00000100:00100000:3.0:1789762792.961237:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ca5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074653440:0@lo:400:kworker.0 00002000:00000024:0.0:1789762792.961244:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 96927744 pending 25559040 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789762792.961264:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762792.961278:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074653440 00000100:00100000:1.0:1789762792.961282:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074653440 00000100:00100000:0.0:1789762792.961285:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074652416:12345-0@lo:13:osp-pre-1-0.0 Request processed in 54us (133us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762792.961290:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 346 00000100:00100000:3.0:1789762792.961294:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074653952:0@lo:400:kworker.0 00000100:00100000:0.0:1789762792.961300:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 145 00000100:00100000:2.0:1789762792.961304:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074653824:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762792.961310:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 347 00000100:00100000:1.0:1789762792.961321:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074653440:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762792.961327:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762792.961342:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074653824:12345-0@lo:400:kworker.0 Request processed in 40us (451us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762792.961348:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 346 00000100:00100000:0.0:1789762792.961360:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074653952 00000100:00100000:0.0:1789762792.961364:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074653952 00000100:00100000:1.0:1789762792.961378:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074653440:12345-0@lo:400:kworker.0 Request processed in 59us (111us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762792.961383:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074653184:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762792.961388:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 347 00000100:00100000:0.0:1789762792.961391:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 134 00000100:00100000:3.0:1789762792.961399:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074653824:0@lo:400:kworker.0 00000100:00100000:0.0:1789762792.961402:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074653952:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789762792.961413:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762792.961414:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074654336:0@lo:400:kworker.0 00000100:00080000:1.0:1789762792.961419:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789762792.961424:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762792.961427:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762792.961441:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074653312 00000100:00100000:0.0:1789762792.961443:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074653952:12345-0@lo:400:kworker.0 Request processed in 43us (118us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762792.961445:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074653312 00000100:00100000:0.0:1789762792.961451:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 134 00000100:00100000:3.0:1789762792.961453:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074652416:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762792.961474:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ca5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074653440:0@lo:400:kworker.0 00000100:00100000:1.0:1789762792.961478:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074652928 00000100:00100000:1.0:1789762792.961482:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074652928 00000100:00100000:3.0:1789762792.961489:0:3424:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294967593 gen 2 00000004:00080000:3.0:1789762792.961497:0:3424:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37d2758700, transno 4294967593 00000100:00100000:1.0:1789762792.961507:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 229 00000100:00100000:3.0:1789762792.961515:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074653952:0@lo:400:kworker.0 00000100:00100000:1.0:1789762792.961520:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074653312:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762792.961520:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:2.0:1789762792.961539:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074652928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762792.961560:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074653312:12345-0@lo:400:kworker.0 Request processed in 44us (1066us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762792.961566:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 229 00000100:00100000:2.0:1789762792.961594:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074652928:12345-0@lo:400:kworker.0 Request processed in 61us (380us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762792.961602:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:0.0:1789762792.961610:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ca6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074652928:0@lo:400:kworker.0 00000100:00100000:1.0:1789762792.961625:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074652672:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762792.961638:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ca7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074653312:0@lo:400:kworker.0 00000100:00100000:1.0:1789762792.961645:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074654208:0@lo:400:kworker.0 00000100:00100000:1.0:1789762792.961653:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074654080:0@lo:400:kworker.0 00000100:00100000:3.0:1789762792.961661:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074652288 00000100:00100000:3.0:1789762792.961667:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074652288 00000100:00100000:3.0:1789762792.961713:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:3.0:1789762792.961728:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074652288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762792.961780:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074652288:12345-0@lo:400:kworker.0 Request processed in 56us (1148us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762792.961789:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074652288:0@lo:400:kworker.0 00000100:00100000:3.0:1789762792.961794:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000001:00080000:2.0:1789762792.964161:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967614 is committed 00002000:00010000:3.0:1789762792.964179:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 42991616 -> 44040192 00002000:00010000:3.0:1789762792.964186:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 83977 -> 86025 00010000:00010000:3.0:1789762792.964198:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.964211:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.964221:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.964237:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762792.964242:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.964252:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.964273:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762792.964282:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f1750ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762792.964340:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067332352:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14810us (49630us total) trans 4294967614 rc 0/0 00000100:00100000:3.0:1789762792.964351:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 72 00000100:00100000:3.0:1789762792.964363:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266196475967 time 266196574664 state 0x2 ref 0 00000100:00100000:1.0:1789762792.966869:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762792.966943:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067332480 00000100:00100000:3.0:1789762792.966949:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067332480 00000100:00100000:3.0:1789762792.966984:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266196475967 time 266196574664 state 0x0 ref 1 00000100:00100000:3.0:1789762792.966998:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266246574664 time 266196574664 state 0x1 ref 1 00000100:00100000:1.0:1789762792.999604:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266246574664 time 266246668723 state 0x0 ref 1 00000100:00100000:1.0:1789762792.999637:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067332480:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762792.999687:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 44040192, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x15009 00010000:00010000:1.0:1789762792.999693:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1750f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.999706:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762792.999709:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762792.999714:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1750f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762792.999726:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1750f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762793.010990:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967615 is committed 00002000:00010000:3.0:1789762793.010995:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 44040192 -> 45088768 00002000:00010000:3.0:1789762793.011010:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 86025 -> 88073 00010000:00010000:3.0:1789762793.011024:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1750f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.011038:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1750f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.011049:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1750f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.011058:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.011063:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1750f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.011073:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1750f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.011093:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1750f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.011101:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1750f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.011179:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067332480:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11544us (44309us total) trans 4294967615 rc 0/0 00000100:00100000:3.0:1789762793.011192:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 73 00000100:00100000:3.0:1789762793.011204:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266246574664 time 266246668723 state 0x2 ref 0 00000100:00100000:1.0:1789762793.013494:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.013535:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067332608 00000100:00100000:3.0:1789762793.013544:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067332608 00000100:00080000:1.0:1789762793.013578:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762793.013603:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266246574664 time 266246668723 state 0x0 ref 1 00000100:00100000:3.0:1789762793.013619:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266296668723 time 266246668723 state 0x1 ref 1 00000100:00100000:1.0:1789762793.049696:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266296668723 time 266296761431 state 0x0 ref 1 00000100:00100000:1.0:1789762793.049721:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067332608:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762793.049788:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 45088768, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x15809 00010000:00010000:1.0:1789762793.049795:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.049808:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762793.049812:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762793.049829:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.049841:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762793.060978:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967616 is committed 00002000:00010000:3.0:1789762793.060982:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 45088768 -> 46137344 00002000:00010000:3.0:1789762793.060987:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 88073 -> 90121 00010000:00010000:3.0:1789762793.060998:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.061010:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.061020:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.061028:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.061033:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.061044:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.061063:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.061079:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f1750f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.061147:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067332608:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11433us (47651us total) trans 4294967616 rc 0/0 00000100:00100000:3.0:1789762793.061154:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 74 00000100:00100000:3.0:1789762793.061163:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266296668723 time 266296761431 state 0x2 ref 0 00000100:00100000:0.0:1789762793.063739:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.063776:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067332736 00000100:00100000:3.0:1789762793.063784:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067332736 00000100:00100000:3.0:1789762793.063823:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266296668723 time 266296761431 state 0x0 ref 1 00000100:00100000:3.0:1789762793.063835:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266346761431 time 266296761431 state 0x1 ref 1 00010000:00010000:3.0:1789762793.088062:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762793.099786:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266346761431 time 266346851943 state 0x0 ref 1 00000100:00100000:1.0:1789762793.099810:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067332736:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762793.099858:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 46137344, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x16009 00010000:00010000:1.0:1789762793.099865:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.099877:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762793.099881:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762793.099886:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.099910:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762793.111312:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967617 is committed 00002000:00010000:3.0:1789762793.111340:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 46137344 -> 47185920 00002000:00010000:3.0:1789762793.111345:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 90121 -> 92169 00010000:00010000:3.0:1789762793.111355:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.111365:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.111372:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.111387:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.111390:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.111397:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.111412:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.111418:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1750ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.111487:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067332736:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11683us (47748us total) trans 4294967617 rc 0/0 00000100:00100000:3.0:1789762793.111496:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 75 00000100:00100000:3.0:1789762793.111506:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266346761431 time 266346851943 state 0x2 ref 0 00000100:00100000:0.0:1789762793.113840:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.113885:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067332864 00000100:00100000:3.0:1789762793.113893:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067332864 00000100:00100000:3.0:1789762793.113965:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266346761431 time 266346851943 state 0x0 ref 1 00000100:00100000:3.0:1789762793.113980:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266396851943 time 266346851943 state 0x1 ref 1 00000100:00100000:2.0:1789762793.149956:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266396851943 time 266396990273 state 0x0 ref 1 00000100:00100000:2.0:1789762793.149982:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067332864:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762793.150028:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 47185920, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x16809 00010000:00010000:2.0:1789762793.150035:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175106 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.150048:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:2.0:1789762793.150064:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762793.150069:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175106 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.150080:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175106 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762793.161681:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967618 is committed 00002000:00010000:3.0:1789762793.161735:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 47185920 -> 48234496 00002000:00010000:3.0:1789762793.161745:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 92169 -> 94219 00010000:00010000:3.0:1789762793.161756:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175106 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.161765:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175106 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.161771:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175106 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.161776:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.161779:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175106 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.161786:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175106 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.161799:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175106 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.161804:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f175106 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.161869:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067332864:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11892us (48027us total) trans 4294967618 rc 0/0 00000100:00100000:3.0:1789762793.161880:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 76 00000100:00100000:3.0:1789762793.161889:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266396851943 time 266396990273 state 0x2 ref 0 00000100:00100000:1.0:1789762793.164357:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.164392:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067332992 00000100:00100000:3.0:1789762793.164400:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067332992 00000100:00100000:3.0:1789762793.164435:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266396851943 time 266396990273 state 0x0 ref 1 00000100:00100000:3.0:1789762793.164446:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266446990273 time 266396990273 state 0x1 ref 1 00000100:00100000:1.0:1789762793.200013:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266446990273 time 266447079215 state 0x0 ref 1 00000100:00100000:1.0:1789762793.200036:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067332992:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762793.200096:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 48234496, mtime: 0x6aad9ce8, atime: 0x0, ctime: 0x6aad9ce8, blocks: 0x1700b 00010000:00010000:1.0:1789762793.200103:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17510d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.200116:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762793.200119:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762793.200124:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17510d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.200135:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17510d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762793.210746:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967619 is committed 00002000:00010000:3.0:1789762793.210765:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 48234496 -> 49283072 00002000:00010000:3.0:1789762793.210770:0:8058:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb mtime from disk: 1789762792 -> 1789762793 00002000:00010000:3.0:1789762793.210773:0:8058:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb ctime from disk: 1789762792 -> 1789762793 00002000:00010000:3.0:1789762793.210775:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 94219 -> 96267 00010000:00010000:3.0:1789762793.210784:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17510d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.210794:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17510d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.210800:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17510d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.210805:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.210808:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17510d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.210814:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17510d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.210843:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17510d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.210852:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17510d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.210956:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067332992:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10927us (46597us total) trans 4294967619 rc 0/0 00000100:00100000:3.0:1789762793.210968:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 77 00000100:00100000:3.0:1789762793.210979:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266446990273 time 266447079215 state 0x2 ref 0 00000100:00100000:0.0:1789762793.213188:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.213222:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067333120 00000100:00100000:3.0:1789762793.213230:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067333120 00000100:00100000:3.0:1789762793.213265:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266446990273 time 266447079215 state 0x0 ref 1 00000100:00100000:3.0:1789762793.213276:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266497079215 time 266447079215 state 0x1 ref 1 00000100:00100000:2.0:1789762793.250096:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266497079215 time 266497157963 state 0x0 ref 1 00000100:00100000:2.0:1789762793.250135:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067333120:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762793.250187:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 49283072, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1780b 00010000:00010000:2.0:1789762793.250194:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175114 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.250208:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:2.0:1789762793.250211:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762793.250215:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175114 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.250226:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175114 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762793.261843:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967620 is committed 00002000:00010000:3.0:1789762793.261863:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 49283072 -> 50331648 00002000:00010000:3.0:1789762793.261869:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 96267 -> 98315 00010000:00010000:3.0:1789762793.261883:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175114 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.261910:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175114 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.261954:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175114 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.261962:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.261967:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175114 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.261977:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175114 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.261997:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175114 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.262006:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c554adc2/0xdf5a7ac59f175114 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.262078:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067333120:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11951us (48887us total) trans 4294967620 rc 0/0 00000100:00100000:3.0:1789762793.262089:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 78 00000100:00100000:3.0:1789762793.262102:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266497079215 time 266497157963 state 0x2 ref 0 00000100:00100000:0.0:1789762793.264300:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.264354:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067333248 00000100:00100000:3.0:1789762793.264363:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067333248 00000100:00100000:3.0:1789762793.264406:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266497079215 time 266497157963 state 0x0 ref 1 00000100:00100000:3.0:1789762793.264432:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266547157963 time 266497157963 state 0x1 ref 1 00000100:00100000:1.0:1789762793.300180:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266547157963 time 266547241948 state 0x0 ref 1 00000100:00100000:1.0:1789762793.300207:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067333248:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762793.300261:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 50331648, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1800b 00010000:00010000:1.0:1789762793.300271:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17511b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.300288:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762793.300293:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762793.300300:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17511b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.300315:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17511b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762793.313109:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967621 is committed 00002000:00010000:3.0:1789762793.313123:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 50331648 -> 51380224 00002000:00010000:3.0:1789762793.313129:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 98315 -> 100363 00010000:00010000:3.0:1789762793.313140:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17511b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.313151:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17511b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.313157:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17511b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.313162:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.313166:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17511b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.313173:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17511b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.313188:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17511b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.313193:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17511b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.313275:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067333248:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13075us (48974us total) trans 4294967621 rc 0/0 00000100:00100000:3.0:1789762793.313284:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 79 00000100:00100000:3.0:1789762793.313295:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266547157963 time 266547241948 state 0x2 ref 0 00000100:00100000:1.0:1789762793.315769:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.315805:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067333376 00000100:00100000:3.0:1789762793.315813:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067333376 00000100:00100000:3.0:1789762793.315862:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266547157963 time 266547241948 state 0x0 ref 1 00000100:00100000:3.0:1789762793.315875:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266597241948 time 266547241948 state 0x1 ref 1 00000100:00100000:1.0:1789762793.350306:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266597241948 time 266597355319 state 0x0 ref 1 00000100:00100000:1.0:1789762793.350331:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067333376:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762793.350381:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 51380224, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1880b 00010000:00010000:1.0:1789762793.350389:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175122 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.350403:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762793.350407:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762793.350411:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175122 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.350423:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175122 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762793.361940:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967622 is committed 00002000:00010000:3.0:1789762793.361962:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 51380224 -> 52428800 00002000:00010000:3.0:1789762793.361967:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 100363 -> 102411 00010000:00010000:3.0:1789762793.361979:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175122 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.361989:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175122 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.361996:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175122 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.362001:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.362004:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175122 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.362018:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175122 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.362033:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175122 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.362038:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175122 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.362101:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067333376:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11778us (46331us total) trans 4294967622 rc 0/0 00000100:00100000:3.0:1789762793.362109:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 80 00000100:00100000:3.0:1789762793.362119:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266597241948 time 266597355319 state 0x2 ref 0 00000100:00100000:0.0:1789762793.364612:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.364649:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067333504 00000100:00100000:3.0:1789762793.364657:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067333504 00000100:00100000:3.0:1789762793.364697:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266597241948 time 266597355319 state 0x0 ref 1 00000100:00100000:3.0:1789762793.364709:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266647355319 time 266597355319 state 0x1 ref 1 00000100:00100000:1.0:1789762793.400367:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266647355319 time 266647434140 state 0x0 ref 1 00000100:00100000:1.0:1789762793.400388:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067333504:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762793.400434:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 52428800, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1900b 00010000:00010000:1.0:1789762793.400441:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f175129 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.400468:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762793.400472:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762793.400476:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f175129 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.400487:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f175129 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762793.412992:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967623 is committed 00002000:00010000:3.0:1789762793.413008:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 52428800 -> 53477376 00002000:00010000:3.0:1789762793.413014:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 102411 -> 104459 00010000:00010000:3.0:1789762793.413024:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f175129 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.413044:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f175129 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.413051:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f175129 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.413056:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.413059:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f175129 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.413066:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f175129 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.413080:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f175129 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.413085:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f175129 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.413142:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067333504:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12761us (48532us total) trans 4294967623 rc 0/0 00000100:00100000:3.0:1789762793.413151:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 81 00000100:00100000:3.0:1789762793.413161:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266647355319 time 266647434140 state 0x2 ref 0 00000100:00100000:1.0:1789762793.415627:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.415661:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067333632 00000100:00100000:3.0:1789762793.415673:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067333632 00000100:00100000:3.0:1789762793.415725:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266647355319 time 266647434140 state 0x0 ref 1 00000100:00100000:3.0:1789762793.415737:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266697434140 time 266647434140 state 0x1 ref 1 00000100:00100000:1.0:1789762793.450482:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266697434140 time 266697541884 state 0x0 ref 1 00000100:00100000:1.0:1789762793.450530:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067333632:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762793.450586:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 53477376, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1980b 00010000:00010000:1.0:1789762793.450595:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175130 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.450612:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762793.450616:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762793.450623:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175130 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.450638:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175130 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762793.464538:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967624 is committed 00002000:00010000:3.0:1789762793.464591:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 53477376 -> 54525952 00002000:00010000:3.0:1789762793.464596:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 104459 -> 106507 00010000:00010000:3.0:1789762793.464606:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175130 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.464616:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175130 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.464622:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175130 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.464627:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.464630:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175130 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.464637:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175130 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.464652:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175130 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.464657:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175130 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.464729:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067333632:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14226us (49102us total) trans 4294967624 rc 0/0 00000100:00100000:3.0:1789762793.464738:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 82 00000100:00100000:3.0:1789762793.464747:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266697434140 time 266697541884 state 0x2 ref 0 00000100:00100000:1.0:1789762793.467516:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.467572:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067333760 00000100:00100000:3.0:1789762793.467582:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067333760 00000100:00100000:3.0:1789762793.467623:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266697434140 time 266697541884 state 0x0 ref 1 00000100:00100000:3.0:1789762793.467635:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266747541884 time 266697541884 state 0x1 ref 1 00000100:00100000:2.0:1789762793.500585:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266747541884 time 266747649972 state 0x0 ref 1 00000100:00100000:2.0:1789762793.500630:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067333760:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762793.500689:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 54525952, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1a00b 00010000:00010000:2.0:1789762793.500701:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175137 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.500721:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:2.0:1789762793.500727:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762793.500734:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175137 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.500752:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175137 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762793.513273:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 54525952 -> 55574528 00000001:00080000:3.0:1789762793.513276:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967625 is committed 00002000:00010000:2.0:1789762793.513280:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 106507 -> 108555 00010000:00010000:2.0:1789762793.513292:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175137 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.513302:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175137 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.513308:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175137 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.513313:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762793.513322:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175137 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.513330:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175137 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.513345:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175137 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.513350:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175137 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762793.513420:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067333760:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12798us (45905us total) trans 4294967625 rc 0/0 00000100:00100000:2.0:1789762793.513428:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 83 00000100:00100000:2.0:1789762793.513438:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266747541884 time 266747649972 state 0x2 ref 0 00000100:00100000:1.0:1789762793.516104:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.516146:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067333888 00000100:00100000:3.0:1789762793.516155:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067333888 00000100:00100000:3.0:1789762793.516195:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266747541884 time 266747649972 state 0x0 ref 1 00000100:00100000:3.0:1789762793.516207:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266797649972 time 266747649972 state 0x1 ref 1 00000100:00100000:1.0:1789762793.550686:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266797649972 time 266797746369 state 0x0 ref 1 00000100:00100000:1.0:1789762793.550716:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067333888:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762793.550770:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 55574528, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1a80b 00010000:00010000:1.0:1789762793.550780:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f17513e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.550798:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762793.550803:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762793.550811:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f17513e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.550828:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f17513e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762793.562794:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967626 is committed 00002000:00010000:3.0:1789762793.562814:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 55574528 -> 56623104 00002000:00010000:3.0:1789762793.562820:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 108555 -> 110603 00010000:00010000:3.0:1789762793.562829:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f17513e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.562839:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f17513e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.562853:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f17513e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.562859:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.562863:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f17513e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.562869:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f17513e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.562884:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f17513e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.562904:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f17513e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.562993:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067333888:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12286us (46888us total) trans 4294967626 rc 0/0 00000100:00100000:3.0:1789762793.563002:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 84 00000100:00100000:3.0:1789762793.563011:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266797649972 time 266797746369 state 0x2 ref 0 00000100:00100000:0.0:1789762793.565543:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.565589:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067334016 00000100:00100000:3.0:1789762793.565600:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067334016 00000100:00100000:3.0:1789762793.565649:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266797649972 time 266797746369 state 0x0 ref 1 00000100:00100000:3.0:1789762793.565681:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266847746369 time 266797746369 state 0x1 ref 1 00000100:00100000:2.0:1789762793.600778:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266847746369 time 266847843460 state 0x0 ref 1 00000100:00100000:2.0:1789762793.600802:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067334016:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762793.600861:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 56623104, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1b00b 00010000:00010000:2.0:1789762793.600887:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175145 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.600946:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:2.0:1789762793.600978:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762793.600988:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175145 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.601004:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175145 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762793.615436:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967627 is committed 00002000:00010000:3.0:1789762793.615530:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 56623104 -> 57671680 00002000:00010000:3.0:1789762793.615536:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 110603 -> 112651 00010000:00010000:3.0:1789762793.615549:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175145 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.615561:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175145 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.615568:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175145 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.615575:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.615579:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175145 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.615588:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175145 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.615604:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175145 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.615611:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175145 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.615703:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067334016:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14907us (50159us total) trans 4294967627 rc 0/0 00000100:00100000:3.0:1789762793.615715:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 85 00000100:00100000:3.0:1789762793.615727:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266847746369 time 266847843460 state 0x2 ref 0 00000100:00100000:0.0:1789762793.618642:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762793.618699:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067334144 00000100:00100000:2.0:1789762793.618708:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067334144 00000100:00100000:2.0:1789762793.618752:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266847746369 time 266847843460 state 0x0 ref 1 00000100:00100000:2.0:1789762793.618765:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266897843460 time 266847843460 state 0x1 ref 1 00000100:00100000:1.0:1789762793.650881:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266897843460 time 266897948438 state 0x0 ref 1 00000100:00100000:1.0:1789762793.650915:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067334144:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762793.650993:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 57671680, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1b80b 00010000:00010000:1.0:1789762793.651010:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17514c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.651024:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762793.651027:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762793.651032:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17514c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.651043:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17514c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762793.662299:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967628 is committed 00002000:00010000:3.0:1789762793.662369:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 57671680 -> 58720256 00002000:00010000:3.0:1789762793.662375:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 112651 -> 114699 00010000:00010000:3.0:1789762793.662389:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17514c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.662401:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17514c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.662408:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17514c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.662414:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.662417:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17514c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.662423:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17514c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.662438:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17514c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.662454:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17514c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.662524:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067334144:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11615us (43880us total) trans 4294967628 rc 0/0 00000100:00100000:3.0:1789762793.662532:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 86 00000100:00100000:3.0:1789762793.662542:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266897843460 time 266897948438 state 0x2 ref 0 00000100:00100000:1.0:1789762793.665138:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.665171:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067334272 00000100:00100000:3.0:1789762793.665179:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067334272 00000100:00100000:3.0:1789762793.665226:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266897843460 time 266897948438 state 0x0 ref 1 00000100:00100000:3.0:1789762793.665241:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266947948438 time 266897948438 state 0x1 ref 1 00000100:00100000:1.0:1789762793.700991:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266947948438 time 266948054507 state 0x0 ref 1 00000100:00100000:1.0:1789762793.701019:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067334272:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762793.701073:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 58720256, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1c00b 00010000:00010000:1.0:1789762793.701081:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f175153 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.701096:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:1.0:1789762793.701101:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762793.701107:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f175153 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.701121:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f175153 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762793.713098:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967629 is committed 00002000:00010000:3.0:1789762793.713146:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 58720256 -> 59768832 00002000:00010000:3.0:1789762793.713151:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 114699 -> 116747 00010000:00010000:3.0:1789762793.713163:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f175153 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.713173:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f175153 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.713179:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f175153 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.713195:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.713198:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f175153 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.713205:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f175153 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.713220:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f175153 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.713225:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f175153 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.713301:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067334272:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12290us (48161us total) trans 4294967629 rc 0/0 00000100:00100000:3.0:1789762793.713311:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 87 00000100:00100000:3.0:1789762793.713320:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266947948438 time 266948054507 state 0x2 ref 0 00000100:00100000:0.0:1789762793.715648:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.715692:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067334400 00000100:00100000:3.0:1789762793.715699:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067334400 00000100:00100000:3.0:1789762793.715737:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266947948438 time 266948054507 state 0x0 ref 1 00000100:00100000:3.0:1789762793.715748:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266998054507 time 266948054507 state 0x1 ref 1 00000100:00100000:3.0:1789762793.751041:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266998054507 time 266998110686 state 0x0 ref 1 00000100:00100000:3.0:1789762793.751069:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067334400:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762793.751108:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 59768832, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1c80b 00010000:00010000:3.0:1789762793.751115:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17515a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.751127:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=266s v=(0 0) 00010000:00010000:3.0:1789762793.751132:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762793.751136:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17515a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.751146:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17515a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762793.761875:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967630 is committed 00002000:00010000:3.0:1789762793.761902:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 59768832 -> 60817408 00002000:00010000:3.0:1789762793.761907:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 116747 -> 118795 00010000:00010000:3.0:1789762793.761932:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17515a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.761942:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17515a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.761948:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17515a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.761954:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.761963:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17515a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.761970:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17515a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.761986:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17515a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.761991:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17515a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.762050:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067334400:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10983us (46401us total) trans 4294967630 rc 0/0 00000100:00100000:3.0:1789762793.762057:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 88 00000100:00100000:3.0:1789762793.762066:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266998054507 time 266998110686 state 0x2 ref 0 00000100:00100000:1.0:1789762793.764163:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.764196:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067334528 00000100:00100000:3.0:1789762793.764203:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067334528 00000100:00100000:3.0:1789762793.764236:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 266998054507 time 266998110686 state 0x0 ref 1 00000100:00100000:3.0:1789762793.764250:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267048110686 time 266998110686 state 0x1 ref 1 00000100:00100000:1.0:1789762793.801117:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267048110686 time 267048184545 state 0x0 ref 1 00000100:00100000:1.0:1789762793.801138:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067334528:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762793.801181:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 60817408, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1d00b 00010000:00010000:1.0:1789762793.801190:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175161 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.801204:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762793.801208:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762793.801215:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175161 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.801229:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175161 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762793.812041:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967631 is committed 00002000:00010000:3.0:1789762793.812104:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 60817408 -> 61865984 00002000:00010000:3.0:1789762793.812108:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 118795 -> 120843 00010000:00010000:3.0:1789762793.812118:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175161 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.812132:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175161 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.812139:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175161 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.812144:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.812147:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175161 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.812153:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175161 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.812166:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175161 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.812171:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175161 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.812228:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067334528:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11094us (48064us total) trans 4294967631 rc 0/0 00000100:00100000:3.0:1789762793.812235:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 89 00000100:00100000:3.0:1789762793.812247:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267048110686 time 267048184545 state 0x2 ref 0 00000100:00100000:2.0:1789762793.815142:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.815203:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067334656 00000100:00100000:3.0:1789762793.815221:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067334656 00000100:00100000:3.0:1789762793.815276:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267048110686 time 267048184545 state 0x0 ref 1 00000100:00100000:3.0:1789762793.815293:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267098184545 time 267048184545 state 0x1 ref 1 00000100:00100000:2.0:1789762793.851216:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267098184545 time 267098281496 state 0x0 ref 1 00000100:00100000:2.0:1789762793.851244:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067334656:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762793.851293:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 61865984, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1d80b 00010000:00010000:2.0:1789762793.851305:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175168 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.851319:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:2.0:1789762793.851322:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762793.851327:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175168 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.851352:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175168 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762793.862969:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967632 is committed 00002000:00010000:2.0:1789762793.862970:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 61865984 -> 62914560 00002000:00010000:2.0:1789762793.862979:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 120843 -> 122891 00010000:00010000:2.0:1789762793.862992:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175168 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.863006:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175168 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.863018:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175168 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.863027:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762793.863032:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175168 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.863041:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175168 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.863058:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175168 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.863063:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175168 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762793.863134:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067334656:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11902us (47994us total) trans 4294967632 rc 0/0 00000100:00100000:2.0:1789762793.863145:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 90 00000100:00100000:2.0:1789762793.863157:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267098184545 time 267098281496 state 0x2 ref 0 00000100:00100000:1.0:1789762793.865646:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.865694:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067334784 00000100:00100000:3.0:1789762793.865703:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067334784 00000100:00100000:3.0:1789762793.865748:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267098184545 time 267098281496 state 0x0 ref 1 00000100:00100000:3.0:1789762793.865763:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267148281496 time 267098281496 state 0x1 ref 1 00000100:00100000:1.0:1789762793.901320:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267148281496 time 267148383252 state 0x0 ref 1 00000100:00100000:1.0:1789762793.901348:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067334784:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762793.901397:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 62914560, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1e00b 00010000:00010000:1.0:1789762793.901418:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17516f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.901433:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762793.901438:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762793.901446:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17516f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762793.901460:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17516f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762793.913158:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967633 is committed 00002000:00010000:3.0:1789762793.913176:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 62914560 -> 63963136 00002000:00010000:3.0:1789762793.913181:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 122891 -> 124939 00010000:00010000:3.0:1789762793.913192:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17516f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.913202:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17516f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.913208:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17516f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.913213:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.913217:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17516f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.913234:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17516f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.913252:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17516f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.913260:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f17516f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.913339:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067334784:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11998us (47691us total) trans 4294967633 rc 0/0 00000100:00100000:3.0:1789762793.913347:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 91 00000100:00100000:3.0:1789762793.913357:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267148281496 time 267148383252 state 0x2 ref 0 00000100:00100000:0.0:1789762793.915841:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762793.915876:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067334912 00000100:00100000:3.0:1789762793.915883:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067334912 00000100:00100000:3.0:1789762793.915947:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267148281496 time 267148383252 state 0x0 ref 1 00000100:00100000:3.0:1789762793.915958:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267198383252 time 267148383252 state 0x1 ref 1 00000100:00100000:2.0:1789762793.951421:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267198383252 time 267198489701 state 0x0 ref 1 00000100:00100000:2.0:1789762793.951440:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067334912:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762793.951479:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 63963136, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1e80b 00010000:00010000:2.0:1789762793.951486:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175176 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.951498:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:2.0:1789762793.951501:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762793.951506:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175176 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762793.951517:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175176 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762793.963486:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967634 is committed 00002000:00010000:3.0:1789762793.963491:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 63963136 -> 65011712 00002000:00010000:3.0:1789762793.963497:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 124939 -> 126987 00010000:00010000:3.0:1789762793.963506:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175176 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.963526:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175176 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.963533:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175176 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.963538:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762793.963541:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175176 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.963551:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175176 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.963571:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175176 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762793.963581:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c05a0c38/0xdf5a7ac59f175176 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762793.963643:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067334912:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12206us (47803us total) trans 4294967634 rc 0/0 00000100:00100000:3.0:1789762793.963651:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 92 00000100:00100000:3.0:1789762793.963660:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267198383252 time 267198489701 state 0x2 ref 0 00000100:00100000:0.0:1789762793.965977:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762793.966028:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067335040 00000100:00100000:2.0:1789762793.966035:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067335040 00000100:00100000:2.0:1789762793.966075:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267198383252 time 267198489701 state 0x0 ref 1 00000100:00100000:2.0:1789762793.966093:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267248489701 time 267198489701 state 0x1 ref 1 00000100:00100000:1.0:1789762794.001496:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267248489701 time 267248561384 state 0x0 ref 1 00000100:00100000:1.0:1789762794.001518:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067335040:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.001560:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 65011712, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1f00b 00010000:00010000:1.0:1789762794.001568:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17517d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.001582:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762794.001586:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.001593:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17517d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.001606:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17517d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762794.012915:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967635 is committed 00002000:00010000:3.0:1789762794.012947:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 65011712 -> 66060288 00002000:00010000:3.0:1789762794.012958:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 126987 -> 129035 00010000:00010000:3.0:1789762794.012968:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17517d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.012976:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17517d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.012982:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17517d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.012987:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.012991:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17517d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.012998:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17517d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.013011:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17517d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.013017:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f17517d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.013069:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067335040:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11558us (47092us total) trans 4294967635 rc 0/0 00000100:00100000:3.0:1789762794.013076:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 93 00000100:00100000:3.0:1789762794.013086:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267248489701 time 267248561384 state 0x2 ref 0 00000100:00100000:0.0:1789762794.015335:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.015370:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067335168 00000100:00100000:3.0:1789762794.015377:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067335168 00000100:00100000:3.0:1789762794.015411:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267248489701 time 267248561384 state 0x0 ref 1 00000100:00100000:3.0:1789762794.015421:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267298561384 time 267248561384 state 0x1 ref 1 00000100:00100000:1.0:1789762794.051585:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267298561384 time 267298647258 state 0x0 ref 1 00000100:00100000:1.0:1789762794.051610:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067335168:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.051660:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 66060288, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x1f80b 00010000:00010000:1.0:1789762794.051670:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175184 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.051686:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762794.051690:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.051696:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175184 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.051711:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175184 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762794.063117:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967636 is committed 00002000:00010000:3.0:1789762794.063142:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 66060288 -> 67108864 00002000:00010000:3.0:1789762794.063148:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 129035 -> 131083 00010000:00010000:3.0:1789762794.063159:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175184 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.063168:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175184 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.063175:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175184 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.063180:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.063184:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175184 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.063190:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175184 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.063206:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175184 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.063222:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175184 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.063290:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067335168:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11687us (47955us total) trans 4294967636 rc 0/0 00000100:00100000:3.0:1789762794.063298:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 94 00000100:00100000:3.0:1789762794.063308:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267298561384 time 267298647258 state 0x2 ref 0 00000100:00100000:0.0:1789762794.065738:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.065773:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067335296 00000100:00100000:3.0:1789762794.065782:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067335296 00000100:00100000:3.0:1789762794.065824:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267298561384 time 267298647258 state 0x0 ref 1 00000100:00100000:3.0:1789762794.065840:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267348647258 time 267298647258 state 0x1 ref 1 00000100:00100000:3.0:1789762794.101658:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267348647258 time 267348714804 state 0x0 ref 1 00000100:00100000:3.0:1789762794.101687:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067335296:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762794.101746:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 67108864, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x2000b 00010000:00010000:3.0:1789762794.101756:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17518b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.101775:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:3.0:1789762794.101781:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762794.101800:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17518b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.101816:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17518b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.112018:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762794.114668:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967637 is committed 00002000:00010000:3.0:1789762794.114691:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 67108864 -> 68157440 00002000:00010000:3.0:1789762794.114698:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 131083 -> 133131 00010000:00010000:3.0:1789762794.114713:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17518b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.114727:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17518b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.114738:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17518b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.114747:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.114752:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17518b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.114762:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17518b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.114784:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17518b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.114808:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f17518b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.114948:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067335296:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13235us (49174us total) trans 4294967637 rc 0/0 00000100:00100000:3.0:1789762794.114961:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 95 00000100:00100000:3.0:1789762794.114976:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267348647258 time 267348714804 state 0x2 ref 0 00000100:00100000:2.0:1789762794.117540:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.117575:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067335424 00000100:00100000:3.0:1789762794.117584:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067335424 00000100:00100000:3.0:1789762794.117624:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267348647258 time 267348714804 state 0x0 ref 1 00000100:00100000:3.0:1789762794.117646:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267398714804 time 267348714804 state 0x1 ref 1 00000100:00100000:1.0:1789762794.151722:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267398714804 time 267398791313 state 0x0 ref 1 00000100:00100000:1.0:1789762794.151741:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067335424:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.151784:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 68157440, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x2080b 00010000:00010000:1.0:1789762794.151791:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175192 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.151804:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762794.151815:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.151820:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175192 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.151832:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175192 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762794.163912:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967638 is committed 00002000:00010000:3.0:1789762794.163914:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 68157440 -> 69206016 00002000:00010000:3.0:1789762794.163939:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 133131 -> 135179 00010000:00010000:3.0:1789762794.163950:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175192 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.163960:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175192 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.163967:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175192 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.163983:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.163987:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175192 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.163993:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175192 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.164008:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175192 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.164013:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175192 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.164073:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067335424:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12336us (46532us total) trans 4294967638 rc 0/0 00000100:00100000:3.0:1789762794.164082:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 96 00000100:00100000:3.0:1789762794.164092:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267398714804 time 267398791313 state 0x2 ref 0 00000100:00100000:0.0:1789762794.166368:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.166405:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067335552 00000100:00100000:3.0:1789762794.166414:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067335552 00000100:00100000:3.0:1789762794.166461:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267398714804 time 267398791313 state 0x0 ref 1 00000100:00100000:3.0:1789762794.166472:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267448791313 time 267398791313 state 0x1 ref 1 00000100:00100000:2.0:1789762794.201815:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267448791313 time 267448883781 state 0x0 ref 1 00000100:00100000:2.0:1789762794.201836:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067335552:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762794.201904:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 69206016, mtime: 0x6aad9ce9, atime: 0x0, ctime: 0x6aad9ce9, blocks: 0x2100b 00010000:00010000:2.0:1789762794.201911:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f175199 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762794.201956:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:2.0:1789762794.201960:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762794.201965:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f175199 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762794.201976:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f175199 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762794.213705:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967639 is committed 00002000:00010000:3.0:1789762794.213706:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 69206016 -> 70254592 00002000:00010000:3.0:1789762794.213717:0:8059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb mtime from disk: 1789762793 -> 1789762794 00002000:00010000:3.0:1789762794.213720:0:8059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb ctime from disk: 1789762793 -> 1789762794 00002000:00010000:3.0:1789762794.213722:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 135179 -> 137227 00010000:00010000:3.0:1789762794.213732:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f175199 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.213742:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f175199 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.213749:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f175199 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.213754:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.213757:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f175199 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.213763:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f175199 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.213777:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f175199 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.213783:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c2b0c0a/0xdf5a7ac59f175199 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.213850:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067335552:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12019us (47479us total) trans 4294967639 rc 0/0 00000100:00100000:3.0:1789762794.213866:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 97 00000100:00100000:3.0:1789762794.213876:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267448791313 time 267448883781 state 0x2 ref 0 00000100:00100000:0.0:1789762794.216335:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.216370:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067335680 00000100:00100000:3.0:1789762794.216379:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067335680 00000100:00100000:3.0:1789762794.216418:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267448791313 time 267448883781 state 0x0 ref 1 00000100:00100000:3.0:1789762794.216429:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267498883781 time 267448883781 state 0x1 ref 1 00000100:00100000:1.0:1789762794.251903:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267498883781 time 267498962696 state 0x0 ref 1 00000100:00100000:1.0:1789762794.251958:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067335680:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.251999:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 70254592, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2180b 00010000:00010000:1.0:1789762794.252006:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f1751a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.252019:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762794.252022:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.252027:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f1751a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.252038:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f1751a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762794.263045:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967640 is committed 00002000:00010000:3.0:1789762794.263109:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 70254592 -> 71303168 00002000:00010000:3.0:1789762794.263113:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 137227 -> 139275 00010000:00010000:3.0:1789762794.263122:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f1751a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.263131:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f1751a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.263138:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f1751a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.263143:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.263146:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f1751a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.263153:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f1751a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.263174:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f1751a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.263179:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f1751a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.263231:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067335680:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11313us (46896us total) trans 4294967640 rc 0/0 00000100:00100000:3.0:1789762794.263239:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 98 00000100:00100000:3.0:1789762794.263248:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267498883781 time 267498962696 state 0x2 ref 0 00000100:00100000:0.0:1789762794.265483:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.265520:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067335808 00000100:00100000:3.0:1789762794.265528:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067335808 00000100:00100000:3.0:1789762794.265563:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267498883781 time 267498962696 state 0x0 ref 1 00000100:00100000:3.0:1789762794.265574:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267548962696 time 267498962696 state 0x1 ref 1 00000100:00100000:2.0:1789762794.302014:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267548962696 time 267549073616 state 0x0 ref 1 00000100:00100000:2.0:1789762794.302062:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067335808:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762794.302116:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 71303168, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2200b 00010000:00010000:2.0:1789762794.302126:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f1751a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762794.302143:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:2.0:1789762794.302147:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762794.302154:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f1751a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762794.302167:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f1751a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762794.314401:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967641 is committed 00002000:00010000:3.0:1789762794.314462:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 71303168 -> 72351744 00002000:00010000:3.0:1789762794.314467:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 139275 -> 141323 00010000:00010000:3.0:1789762794.314477:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f1751a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.314487:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f1751a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.314504:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f1751a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.314509:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.314513:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f1751a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.314519:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f1751a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.314534:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f1751a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.314540:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2f56565/0xdf5a7ac59f1751a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.314610:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067335808:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12555us (49125us total) trans 4294967641 rc 0/0 00000100:00100000:3.0:1789762794.314619:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 99 00000100:00100000:3.0:1789762794.314628:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267548962696 time 267549073616 state 0x2 ref 0 00000100:00100000:0.0:1789762794.317050:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.317083:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067335936 00000100:00100000:3.0:1789762794.317092:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067335936 00000100:00100000:3.0:1789762794.317129:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267548962696 time 267549073616 state 0x0 ref 1 00000100:00100000:3.0:1789762794.317149:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267599073616 time 267549073616 state 0x1 ref 1 00000100:00100000:1.0:1789762794.352092:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267599073616 time 267599155603 state 0x0 ref 1 00000100:00100000:1.0:1789762794.352134:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067335936:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.352189:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 72351744, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2280b 00010000:00010000:1.0:1789762794.352199:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1751ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.352216:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762794.352221:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.352228:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1751ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.352244:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1751ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762794.364021:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967642 is committed 00002000:00010000:3.0:1789762794.364067:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 72351744 -> 73400320 00002000:00010000:3.0:1789762794.364073:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 141323 -> 143371 00010000:00010000:3.0:1789762794.364086:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1751ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.364098:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1751ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.364106:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1751ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.364112:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.364116:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1751ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.364140:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1751ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.364155:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1751ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.364160:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1751ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.364248:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067335936:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12119us (47196us total) trans 4294967642 rc 0/0 00000100:00100000:3.0:1789762794.364258:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 100 00000100:00100000:3.0:1789762794.364267:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267599073616 time 267599155603 state 0x2 ref 0 00000100:00100000:2.0:1789762794.366657:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.366704:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067336064 00000100:00100000:3.0:1789762794.366713:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067336064 00000100:00100000:3.0:1789762794.366751:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267599073616 time 267599155603 state 0x0 ref 1 00000100:00100000:3.0:1789762794.366761:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267649155603 time 267599155603 state 0x1 ref 1 00000100:00100000:1.0:1789762794.402192:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267649155603 time 267649254172 state 0x0 ref 1 00000100:00100000:1.0:1789762794.402222:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067336064:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.402275:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 73400320, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2300b 00010000:00010000:1.0:1789762794.402302:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f1751b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.402321:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762794.402326:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.402334:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f1751b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.402367:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f1751b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762794.416839:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967643 is committed 00002000:00010000:3.0:1789762794.416865:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 73400320 -> 74448896 00002000:00010000:3.0:1789762794.416871:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 143371 -> 145419 00010000:00010000:3.0:1789762794.416882:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f1751b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.416909:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f1751b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.416950:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f1751b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.416957:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.416960:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f1751b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.416974:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f1751b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.417006:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f1751b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.417013:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f1751b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.417083:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067336064:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14870us (50424us total) trans 4294967643 rc 0/0 00000100:00100000:3.0:1789762794.417093:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 101 00000100:00100000:3.0:1789762794.417103:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267649155603 time 267649254172 state 0x2 ref 0 00000100:00100000:0.0:1789762794.419768:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.419805:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067336192 00000100:00100000:3.0:1789762794.419813:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067336192 00000100:00100000:3.0:1789762794.419851:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267649155603 time 267649254172 state 0x0 ref 1 00000100:00100000:3.0:1789762794.419863:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267699254172 time 267649254172 state 0x1 ref 1 00000100:00100000:1.0:1789762794.452373:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267699254172 time 267699441531 state 0x0 ref 1 00000100:00100000:1.0:1789762794.452394:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067336192:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.452454:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 74448896, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2380b 00010000:00010000:1.0:1789762794.452461:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1751bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.452472:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762794.452476:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.452480:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1751bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.452509:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1751bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762794.469010:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967644 is committed 00002000:00010000:3.0:1789762794.469010:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:3.0:1789762794.469019:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 145419 -> 147469 00010000:00010000:3.0:1789762794.469032:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1751bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.469056:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1751bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.469065:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1751bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.469071:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.469076:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1751bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.469084:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1751bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.469100:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1751bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.469107:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f1751bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.469174:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067336192:12345-192.168.203.37@tcp:4:dd.500 Request processed in 16783us (49403us total) trans 4294967644 rc 0/0 00000100:00100000:3.0:1789762794.469188:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 102 00000100:00100000:3.0:1789762794.469203:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267699254172 time 267699441531 state 0x2 ref 0 00000100:00100000:2.0:1789762794.472322:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.472359:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067336320 00000100:00100000:3.0:1789762794.472374:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067336320 00000100:00100000:3.0:1789762794.472412:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267699254172 time 267699441531 state 0x0 ref 1 00000100:00100000:3.0:1789762794.472439:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267749441531 time 267699441531 state 0x1 ref 1 00000100:00100000:1.0:1789762794.502507:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267749441531 time 267749556956 state 0x0 ref 1 00000100:00100000:1.0:1789762794.502534:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067336320:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.502583:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 75497472, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2400d 00010000:00010000:1.0:1789762794.502617:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f1751c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.502630:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762794.502633:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.502638:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f1751c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.502649:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f1751c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762794.516377:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967645 is committed 00002000:00010000:3.0:1789762794.516495:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 75497472 -> 76546048 00002000:00010000:3.0:1789762794.516502:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 147469 -> 149517 00010000:00010000:3.0:1789762794.516516:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f1751c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.516546:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f1751c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.516554:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f1751c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.516561:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.516565:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f1751c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.516574:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f1751c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.516608:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f1751c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.516614:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f1751c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.516694:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067336320:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14166us (44371us total) trans 4294967645 rc 0/0 00000100:00100000:3.0:1789762794.516722:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 103 00000100:00100000:3.0:1789762794.516733:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267749441531 time 267749556956 state 0x2 ref 0 00000100:00100000:0.0:1789762794.519467:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.519501:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067336448 00000100:00100000:3.0:1789762794.519509:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067336448 00000100:00100000:3.0:1789762794.519549:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267749441531 time 267749556956 state 0x0 ref 1 00000100:00100000:3.0:1789762794.519560:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267799556956 time 267749556956 state 0x1 ref 1 00000100:00100000:2.0:1789762794.552564:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267799556956 time 267799630092 state 0x0 ref 1 00000100:00100000:2.0:1789762794.552585:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067336448:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762794.552638:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 76546048, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2480d 00010000:00010000:2.0:1789762794.552645:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1751ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762794.552658:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:2.0:1789762794.552661:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762794.552676:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1751ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762794.552687:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1751ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762794.565501:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967646 is committed 00002000:00010000:3.0:1789762794.565567:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 76546048 -> 77594624 00002000:00010000:3.0:1789762794.565572:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 149517 -> 151565 00010000:00010000:3.0:1789762794.565582:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1751ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.565591:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1751ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.565598:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1751ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.565603:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.565606:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1751ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.565620:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1751ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.565634:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1751ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.565657:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1751ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.565755:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067336448:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13175us (46286us total) trans 4294967646 rc 0/0 00000100:00100000:3.0:1789762794.565766:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 104 00000100:00100000:3.0:1789762794.565796:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267799556956 time 267799630092 state 0x2 ref 0 00000100:00100000:0.0:1789762794.568863:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.569168:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067336576 00000100:00100000:3.0:1789762794.569177:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067336576 00000100:00100000:3.0:1789762794.569277:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267799556956 time 267799630092 state 0x0 ref 1 00000100:00100000:3.0:1789762794.569312:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267849630092 time 267799630092 state 0x1 ref 1 00000100:00100000:1.0:1789762794.602693:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267849630092 time 267849755927 state 0x0 ref 1 00000100:00100000:1.0:1789762794.602721:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067336576:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.602775:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 77594624, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2500d 00010000:00010000:1.0:1789762794.602785:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1751d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.602804:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762794.602810:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.602817:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1751d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.602833:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1751d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762794.622610:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:3.0:1789762794.622618:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 151565 -> 153613 00000001:00080000:1.0:1789762794.622630:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967647 is committed 00010000:00010000:3.0:1789762794.622634:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1751d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.622667:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1751d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.622677:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1751d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.622684:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.622689:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1751d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.622698:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1751d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.622718:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1751d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.622726:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f1751d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.622810:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067336576:12345-192.168.203.37@tcp:4:dd.500 Request processed in 20096us (53947us total) trans 4294967647 rc 0/0 00000100:00100000:3.0:1789762794.622821:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 105 00000100:00100000:3.0:1789762794.622835:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267849630092 time 267849755927 state 0x2 ref 0 00000100:00100000:0.0:1789762794.626777:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.626830:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067336704 00000100:00100000:3.0:1789762794.626848:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067336704 00000100:00100000:3.0:1789762794.626981:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267849630092 time 267849755927 state 0x0 ref 1 00000100:00100000:3.0:1789762794.627023:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267899755927 time 267849755927 state 0x1 ref 1 00000100:00100000:2.0:1789762794.652807:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267899755927 time 267899873243 state 0x0 ref 1 00000100:00100000:2.0:1789762794.652830:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067336704:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762794.652878:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 78643200, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2580d 00010000:00010000:2.0:1789762794.652906:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f1751d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762794.652950:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:2.0:1789762794.652956:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762794.652981:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f1751d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762794.652999:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f1751d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762794.667529:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967648 is committed 00002000:00010000:3.0:1789762794.667599:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:3.0:1789762794.667605:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 153613 -> 155661 00010000:00010000:3.0:1789762794.667617:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f1751d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.667628:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f1751d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.667636:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f1751d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.667642:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.667645:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f1751d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.667653:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f1751d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.667672:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f1751d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.667678:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f1751d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.667763:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067336704:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14938us (40987us total) trans 4294967648 rc 0/0 00000100:00100000:3.0:1789762794.667773:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 106 00000100:00100000:3.0:1789762794.667784:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267899755927 time 267899873243 state 0x2 ref 0 00000100:00100000:0.0:1789762794.670598:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.670641:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067336832 00000100:00100000:3.0:1789762794.670652:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067336832 00000100:00100000:3.0:1789762794.670696:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267899755927 time 267899873243 state 0x0 ref 1 00000100:00100000:3.0:1789762794.670709:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267949873243 time 267899873243 state 0x1 ref 1 00000100:00100000:1.0:1789762794.702966:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267949873243 time 267949995011 state 0x0 ref 1 00000100:00100000:1.0:1789762794.702998:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067336832:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.703056:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 79691776, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2600d 00010000:00010000:1.0:1789762794.703065:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1751df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.703081:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762794.703085:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.703091:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1751df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.703103:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1751df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762794.714851:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967649 is committed 00002000:00010000:3.0:1789762794.714952:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 79691776 -> 80740352 00002000:00010000:3.0:1789762794.714957:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 155661 -> 157709 00010000:00010000:3.0:1789762794.714969:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1751df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.714984:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1751df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.714995:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1751df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.715003:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.715010:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1751df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.715025:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1751df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.715041:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1751df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.715047:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f1751df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.715120:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067336832:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12134us (44522us total) trans 4294967649 rc 0/0 00000100:00100000:3.0:1789762794.715129:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 107 00000100:00100000:3.0:1789762794.715139:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267949873243 time 267949995011 state 0x2 ref 0 00000100:00100000:0.0:1789762794.718010:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.718045:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067336960 00000100:00100000:3.0:1789762794.718054:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067336960 00000100:00100000:3.0:1789762794.718095:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267949873243 time 267949995011 state 0x0 ref 1 00000100:00100000:3.0:1789762794.718107:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267999995011 time 267949995011 state 0x1 ref 1 00000100:00100000:1.0:1789762794.753040:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267999995011 time 268000104647 state 0x0 ref 1 00000100:00100000:1.0:1789762794.753064:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067336960:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.753121:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 80740352, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2680d 00010000:00010000:1.0:1789762794.753131:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.753164:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=267s v=(0 0) 00010000:00010000:1.0:1789762794.753168:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.753175:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.753202:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762794.765152:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967650 is committed 00002000:00010000:3.0:1789762794.765167:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 80740352 -> 81788928 00002000:00010000:3.0:1789762794.765172:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 157709 -> 159757 00010000:00010000:3.0:1789762794.765183:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.765203:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.765209:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.765214:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.765218:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.765225:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.765240:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.765246:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.765305:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067336960:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12249us (47295us total) trans 4294967650 rc 0/0 00000100:00100000:3.0:1789762794.765313:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 108 00000100:00100000:3.0:1789762794.765323:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267999995011 time 268000104647 state 0x2 ref 0 00000100:00100000:2.0:1789762794.767582:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.767623:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067337088 00000100:00100000:3.0:1789762794.767639:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067337088 00000100:00100000:3.0:1789762794.767683:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 267999995011 time 268000104647 state 0x0 ref 1 00000100:00100000:3.0:1789762794.767694:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268050104647 time 268000104647 state 0x1 ref 1 00000100:00100000:1.0:1789762794.803109:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268050104647 time 268050174331 state 0x0 ref 1 00000100:00100000:1.0:1789762794.803153:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067337088:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.803235:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 81788928, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2700d 00010000:00010000:1.0:1789762794.803247:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.803282:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:1.0:1789762794.803287:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.803294:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.803310:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762794.816946:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967651 is committed 00002000:00010000:1.0:1789762794.816957:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 81788928 -> 82837504 00002000:00010000:1.0:1789762794.816968:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 159757 -> 161805 00010000:00010000:1.0:1789762794.816978:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.816988:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.816995:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.817000:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762794.817003:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.817010:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.817024:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.817030:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762794.817084:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067337088:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13942us (49502us total) trans 4294967651 rc 0/0 00000100:00100000:1.0:1789762794.817093:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 109 00000100:00100000:1.0:1789762794.817103:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268050104647 time 268050174331 state 0x2 ref 0 00000100:00100000:2.0:1789762794.819791:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.819857:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067337216 00000100:00100000:3.0:1789762794.819867:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067337216 00000100:00100000:3.0:1789762794.819963:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268050104647 time 268050174331 state 0x0 ref 1 00000100:00100000:3.0:1789762794.819976:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268100174331 time 268050174331 state 0x1 ref 1 00000100:00100000:1.0:1789762794.853195:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268100174331 time 268100261159 state 0x0 ref 1 00000100:00100000:1.0:1789762794.853220:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067337216:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.853268:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 82837504, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2780d 00010000:00010000:1.0:1789762794.853275:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.853287:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:1.0:1789762794.853291:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.853296:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.853306:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762794.864047:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967652 is committed 00002000:00010000:3.0:1789762794.864065:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 82837504 -> 83886080 00002000:00010000:3.0:1789762794.864071:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 161805 -> 163853 00010000:00010000:3.0:1789762794.864084:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.864097:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.864109:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.864117:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.864123:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.864132:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.864157:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.864163:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f1751f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.864220:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067337216:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11009us (44430us total) trans 4294967652 rc 0/0 00000100:00100000:3.0:1789762794.864228:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 110 00000100:00100000:3.0:1789762794.864238:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268100174331 time 268100261159 state 0x2 ref 0 00000100:00100000:0.0:1789762794.866344:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.866387:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067337344 00000100:00100000:3.0:1789762794.866396:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067337344 00000100:00100000:3.0:1789762794.866432:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268100174331 time 268100261159 state 0x0 ref 1 00000100:00100000:3.0:1789762794.866443:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268150261159 time 268100261159 state 0x1 ref 1 00000100:00100000:2.0:1789762794.903269:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268150261159 time 268150337033 state 0x0 ref 1 00000100:00100000:2.0:1789762794.903303:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067337344:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762794.903345:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 83886080, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2800d 00010000:00010000:2.0:1789762794.903352:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f1751fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762794.903365:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:2.0:1789762794.903368:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762794.903372:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f1751fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762794.903383:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f1751fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762794.914462:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967653 is committed 00002000:00010000:3.0:1789762794.914525:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 83886080 -> 84934656 00002000:00010000:3.0:1789762794.914532:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 163853 -> 165901 00010000:00010000:3.0:1789762794.914545:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f1751fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.914559:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f1751fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.914583:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f1751fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.914589:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.914593:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f1751fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.914600:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f1751fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.914616:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f1751fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.914621:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ded696a4/0xdf5a7ac59f1751fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.914688:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067337344:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11388us (48341us total) trans 4294967653 rc 0/0 00000100:00100000:3.0:1789762794.914696:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 111 00000100:00100000:3.0:1789762794.914706:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268150261159 time 268150337033 state 0x2 ref 0 00000100:00100000:0.0:1789762794.917075:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762794.917131:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067337472 00000100:00100000:1.0:1789762794.917140:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067337472 00000100:00100000:1.0:1789762794.917179:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268150261159 time 268150337033 state 0x0 ref 1 00000100:00100000:1.0:1789762794.917190:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268200337033 time 268150337033 state 0x1 ref 1 00000100:00100000:1.0:1789762794.953314:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268200337033 time 268200382042 state 0x0 ref 1 00000100:00100000:1.0:1789762794.953335:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067337472:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762794.953388:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 84934656, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2880d 00010000:00010000:1.0:1789762794.953395:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175202 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.953408:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:1.0:1789762794.953411:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762794.953416:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175202 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762794.953427:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175202 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762794.964964:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967654 is committed 00002000:00010000:3.0:1789762794.964977:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 84934656 -> 85983232 00002000:00010000:3.0:1789762794.964982:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 165901 -> 167949 00010000:00010000:3.0:1789762794.964993:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175202 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.965014:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175202 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.965020:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175202 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.965025:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762794.965029:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175202 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.965036:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175202 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.965050:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175202 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762794.965056:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175202 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762794.965173:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067337472:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11844us (48096us total) trans 4294967654 rc 0/0 00000100:00100000:3.0:1789762794.965182:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 112 00000100:00100000:3.0:1789762794.965191:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268200337033 time 268200382042 state 0x2 ref 0 00000100:00100000:0.0:1789762794.967387:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762794.967428:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067337600 00000100:00100000:3.0:1789762794.967442:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067337600 00000100:00100000:3.0:1789762794.967476:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268200337033 time 268200382042 state 0x0 ref 1 00000100:00100000:3.0:1789762794.967487:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268250382042 time 268200382042 state 0x1 ref 1 00000100:00100000:2.0:1789762795.003399:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268250382042 time 268250465198 state 0x0 ref 1 00000100:00100000:2.0:1789762795.003433:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067337600:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:1.0:1789762795.003463:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762795.003479:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 85983232, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2900d 00010000:00010000:2.0:1789762795.003487:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175209 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.003500:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:2.0:1789762795.003504:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762795.003508:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175209 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.003534:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175209 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762795.010467:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762795.010486:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074654464:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762795.010556:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762795.010590:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074654464 00000100:00100000:1.0:1789762795.010598:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074654464 00000100:00100000:1.0:1789762795.010651:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 231 00000100:00100000:1.0:1789762795.010661:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074654464:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762795.010697:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762795.010971:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074654464:12345-0@lo:41:osp-pre-1-0.0 Request processed in 312us (415us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762795.010979:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 231 00000100:00100000:1.0:1789762795.011044:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074654464:0@lo:41:osp-pre-1-0.0 00002000:00010000:0.0:1789762795.018256:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 85983232 -> 87031808 00002000:00010000:0.0:1789762795.018262:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 167949 -> 169997 00010000:00010000:0.0:1789762795.018276:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175209 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.018287:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175209 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.018294:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175209 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.018300:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762795.018305:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175209 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.018312:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175209 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.018329:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175209 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.018334:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175209 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762795.018441:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2398:x1876702067337600:12345-192.168.203.37@tcp:4:dd.500 Request processed in 15024us (51052us total) trans 4294967655 rc 0/0 00000100:00100000:0.0:1789762795.018452:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 113 00000100:00100000:0.0:1789762795.018462:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268250382042 time 268250465198 state 0x2 ref 0 00000001:00080000:0.0:1789762795.018498:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967655 is committed 00000100:00100000:0.0:1789762795.021163:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762795.021207:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067337728 00000100:00100000:2.0:1789762795.021215:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067337728 00000100:00100000:2.0:1789762795.021258:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268250382042 time 268250465198 state 0x0 ref 1 00000100:00100000:2.0:1789762795.021270:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268300465198 time 268250465198 state 0x1 ref 1 00000001:00080000:1.0:1789762795.023242:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967356 is committed 00000001:00080000:1.0:1789762795.023269:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967361 is committed 00000001:00080000:1.0:1789762795.023296:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967362 is committed 00000100:00100000:3.0:1789762795.053492:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268300465198 time 268300558405 state 0x0 ref 1 00000100:00100000:3.0:1789762795.053516:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067337728:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762795.053564:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 87031808, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2980d 00010000:00010000:3.0:1789762795.053571:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f175210 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.053584:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:3.0:1789762795.053588:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.053592:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f175210 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.053603:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f175210 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762795.067257:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967656 is committed 00002000:00010000:1.0:1789762795.067286:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 87031808 -> 88080384 00002000:00010000:1.0:1789762795.067299:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 169997 -> 172045 00010000:00010000:1.0:1789762795.067312:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f175210 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.067328:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f175210 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.067338:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f175210 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.067347:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.067352:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f175210 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.067361:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f175210 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.067377:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f175210 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.067382:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e3d2b6d/0xdf5a7ac59f175210 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.067463:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067337728:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13953us (46298us total) trans 4294967656 rc 0/0 00000100:00100000:1.0:1789762795.067471:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 114 00000100:00100000:1.0:1789762795.067482:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268300465198 time 268300558405 state 0x2 ref 0 00000100:00100000:3.0:1789762795.070012:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762795.070043:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067337856 00000100:00100000:1.0:1789762795.070051:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067337856 00000100:00100000:1.0:1789762795.070091:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268300465198 time 268300558405 state 0x0 ref 1 00000100:00100000:1.0:1789762795.070102:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268350558405 time 268300558405 state 0x1 ref 1 00000100:00100000:1.0:1789762795.103560:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268350558405 time 268350625806 state 0x0 ref 1 00000100:00100000:1.0:1789762795.103585:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067337856:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762795.103632:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 88080384, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2a00d 00010000:00010000:1.0:1789762795.103639:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175217 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.103652:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:1.0:1789762795.103656:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762795.103660:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175217 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.103671:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175217 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762795.115426:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967657 is committed 00002000:00010000:1.0:1789762795.115436:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 88080384 -> 89128960 00002000:00010000:1.0:1789762795.115441:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 172045 -> 174093 00010000:00010000:1.0:1789762795.115451:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175217 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.115468:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175217 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.115475:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175217 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.115480:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.115484:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175217 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.115490:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175217 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.115505:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175217 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.115510:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175217 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.115575:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067337856:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11999us (45562us total) trans 4294967657 rc 0/0 00000100:00100000:1.0:1789762795.115584:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 115 00000100:00100000:1.0:1789762795.115594:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268350558405 time 268350625806 state 0x2 ref 0 00000100:00100000:3.0:1789762795.117972:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762795.118019:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067337984 00000100:00100000:0.0:1789762795.118029:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067337984 00000100:00100000:0.0:1789762795.118069:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268350558405 time 268350625806 state 0x0 ref 1 00000100:00100000:0.0:1789762795.118081:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268400625806 time 268350625806 state 0x1 ref 1 00010000:00010000:1.0:1789762795.136061:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762795.153675:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268400625806 time 268400740969 state 0x0 ref 1 00000100:00100000:1.0:1789762795.153698:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067337984:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762795.153746:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 89128960, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2a80d 00010000:00010000:1.0:1789762795.153754:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17521e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.153767:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:1.0:1789762795.153770:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762795.153775:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17521e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.153785:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17521e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762795.165808:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967658 is committed 00002000:00010000:1.0:1789762795.165825:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 89128960 -> 90177536 00002000:00010000:1.0:1789762795.165832:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 174093 -> 176141 00010000:00010000:1.0:1789762795.165844:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17521e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.165853:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17521e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.165860:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17521e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.165875:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.165891:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17521e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.165899:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17521e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.165914:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17521e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.165935:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17521e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.165990:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067337984:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12300us (48018us total) trans 4294967658 rc 0/0 00000100:00100000:1.0:1789762795.165999:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 116 00000100:00100000:1.0:1789762795.166008:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268400625806 time 268400740969 state 0x2 ref 0 00000100:00100000:3.0:1789762795.168173:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762795.168211:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067338112 00000100:00100000:1.0:1789762795.168220:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067338112 00000100:00100000:1.0:1789762795.168257:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268400625806 time 268400740969 state 0x0 ref 1 00000100:00100000:1.0:1789762795.168269:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268450740969 time 268400740969 state 0x1 ref 1 00000100:00100000:3.0:1789762795.203788:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268450740969 time 268450854485 state 0x0 ref 1 00000100:00100000:3.0:1789762795.203811:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067338112:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762795.203855:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 90177536, mtime: 0x6aad9cea, atime: 0x0, ctime: 0x6aad9cea, blocks: 0x2b00d 00010000:00010000:3.0:1789762795.203862:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f175225 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.203875:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:3.0:1789762795.203892:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.203898:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f175225 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.203909:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f175225 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762795.214938:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967659 is committed 00002000:00010000:1.0:1789762795.214974:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 90177536 -> 91226112 00002000:00010000:1.0:1789762795.214986:0:13301:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb mtime from disk: 1789762794 -> 1789762795 00002000:00010000:1.0:1789762795.214988:0:13301:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb ctime from disk: 1789762794 -> 1789762795 00002000:00010000:1.0:1789762795.214990:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 176141 -> 178189 00010000:00010000:1.0:1789762795.215000:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f175225 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.215009:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f175225 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.215016:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f175225 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.215021:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.215024:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f175225 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.215030:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f175225 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.215045:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f175225 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.215051:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5eb1579/0xdf5a7ac59f175225 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.215119:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067338112:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11315us (46945us total) trans 4294967659 rc 0/0 00000100:00100000:1.0:1789762795.215135:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 117 00000100:00100000:1.0:1789762795.215145:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268450740969 time 268450854485 state 0x2 ref 0 00000100:00100000:3.0:1789762795.217345:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762795.217425:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067338240 00000100:00100000:1.0:1789762795.217434:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067338240 00000100:00100000:1.0:1789762795.217473:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268450740969 time 268450854485 state 0x0 ref 1 00000100:00100000:1.0:1789762795.217484:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268500854485 time 268450854485 state 0x1 ref 1 00000100:00100000:2.0:1789762795.253900:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268500854485 time 268500953980 state 0x0 ref 1 00000100:00100000:2.0:1789762795.253944:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067338240:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762795.253987:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 91226112, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x2b80d 00010000:00010000:2.0:1789762795.253994:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f17522c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.254007:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:2.0:1789762795.254011:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762795.254016:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f17522c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.254042:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f17522c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762795.266631:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 91226112 -> 92274688 00002000:00010000:2.0:1789762795.266637:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 178189 -> 180237 00010000:00010000:2.0:1789762795.266646:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f17522c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762795.266648:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967660 is committed 00010000:00010000:2.0:1789762795.266659:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f17522c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.266666:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f17522c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.266672:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762795.266675:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f17522c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.266681:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f17522c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.266694:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f17522c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.266699:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e225e648/0xdf5a7ac59f17522c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762795.266760:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067338240:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12845us (49415us total) trans 4294967660 rc 0/0 00000100:00100000:2.0:1789762795.266768:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 118 00000100:00100000:2.0:1789762795.266776:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268500854485 time 268500953980 state 0x2 ref 0 00000100:00100000:3.0:1789762795.268857:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762795.268941:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067338368 00000100:00100000:1.0:1789762795.268950:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067338368 00000100:00100000:1.0:1789762795.268986:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268500854485 time 268500953980 state 0x0 ref 1 00000100:00100000:1.0:1789762795.268998:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268550953980 time 268500953980 state 0x1 ref 1 00000100:00100000:3.0:1789762795.303995:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268550953980 time 268551056166 state 0x0 ref 1 00000100:00100000:3.0:1789762795.304035:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067338368:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762795.304084:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 92274688, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x2c00d 00010000:00010000:3.0:1789762795.304091:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175233 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.304120:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:3.0:1789762795.304123:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.304128:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175233 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.304138:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175233 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762795.317185:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967661 is committed 00002000:00010000:1.0:1789762795.317210:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 92274688 -> 93323264 00002000:00010000:1.0:1789762795.317216:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 180237 -> 182285 00010000:00010000:1.0:1789762795.317231:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175233 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.317246:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175233 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.317254:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175233 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.317270:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.317274:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175233 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.317281:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175233 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.317295:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175233 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.317300:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175233 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.317378:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067338368:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13353us (48518us total) trans 4294967661 rc 0/0 00000100:00100000:1.0:1789762795.317387:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 119 00000100:00100000:1.0:1789762795.317396:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268550953980 time 268551056166 state 0x2 ref 0 00000100:00100000:0.0:1789762795.319737:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762795.319817:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067338496 00000100:00100000:1.0:1789762795.319827:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067338496 00000100:00100000:1.0:1789762795.319873:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268550953980 time 268551056166 state 0x0 ref 1 00000100:00100000:1.0:1789762795.319899:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268601056166 time 268551056166 state 0x1 ref 1 00000100:00100000:0.0:1789762795.354100:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268601056166 time 268601159627 state 0x0 ref 1 00000100:00100000:0.0:1789762795.354129:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067338496:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762795.354194:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 93323264, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x2c80d 00010000:00010000:0.0:1789762795.354203:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f17523a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.354219:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:0.0:1789762795.354234:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762795.354240:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f17523a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.354255:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f17523a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762795.367214:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967662 is committed 00002000:00010000:1.0:1789762795.367235:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 93323264 -> 94371840 00002000:00010000:1.0:1789762795.367248:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 182285 -> 184333 00010000:00010000:1.0:1789762795.367260:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f17523a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.367270:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f17523a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.367277:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f17523a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.367282:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.367285:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f17523a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.367293:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f17523a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.367308:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f17523a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.367313:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005feb0681/0xdf5a7ac59f17523a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.367379:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067338496:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13259us (47646us total) trans 4294967662 rc 0/0 00000100:00100000:1.0:1789762795.367388:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 120 00000100:00100000:1.0:1789762795.367397:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268601056166 time 268601159627 state 0x2 ref 0 00000100:00100000:3.0:1789762795.369461:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762795.369512:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067338624 00000100:00100000:2.0:1789762795.369520:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067338624 00000100:00100000:2.0:1789762795.369559:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268601056166 time 268601159627 state 0x0 ref 1 00000100:00100000:2.0:1789762795.369570:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268651159627 time 268601159627 state 0x1 ref 1 00000100:00100000:3.0:1789762795.404172:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268651159627 time 268651239223 state 0x0 ref 1 00000100:00100000:3.0:1789762795.404194:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff687800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067338624:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762795.404237:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 94371840, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x2d00d 00010000:00010000:3.0:1789762795.404244:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f175241 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.404258:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:3.0:1789762795.404262:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.404266:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f175241 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.404278:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f175241 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762795.417626:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967663 is committed 00002000:00010000:1.0:1789762795.417682:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 94371840 -> 95420416 00002000:00010000:1.0:1789762795.417687:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 184333 -> 186381 00010000:00010000:1.0:1789762795.417696:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f175241 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.417705:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f175241 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.417711:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f175241 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.417716:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.417720:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f175241 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.417727:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f175241 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.417742:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f175241 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.417746:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e89e2018/0xdf5a7ac59f175241 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.417809:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff687800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067338624:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13622us (48347us total) trans 4294967663 rc 0/0 00000100:00100000:1.0:1789762795.417827:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 121 00000100:00100000:1.0:1789762795.417836:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268651159627 time 268651239223 state 0x2 ref 0 00000100:00100000:3.0:1789762795.420047:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762795.420081:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067338752 00000100:00100000:1.0:1789762795.420089:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067338752 00000100:00100000:1.0:1789762795.420125:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268651159627 time 268651239223 state 0x0 ref 1 00000100:00100000:1.0:1789762795.420136:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268701239223 time 268651239223 state 0x1 ref 1 00000100:00100000:3.0:1789762795.454274:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268701239223 time 268701333554 state 0x0 ref 1 00000100:00100000:3.0:1789762795.454300:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff685880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067338752:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762795.454349:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 95420416, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x2d80d 00010000:00010000:3.0:1789762795.454357:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f175248 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.454370:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:3.0:1789762795.454374:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.454378:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f175248 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.454389:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f175248 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762795.467081:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967664 is committed 00002000:00010000:1.0:1789762795.467160:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 95420416 -> 96468992 00002000:00010000:1.0:1789762795.467167:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 186381 -> 188429 00010000:00010000:1.0:1789762795.467181:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f175248 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.467195:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f175248 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.467205:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f175248 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.467214:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.467219:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f175248 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.467228:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f175248 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.467257:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f175248 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.467265:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1721668/0xdf5a7ac59f175248 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.467339:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff685880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067338752:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13047us (47290us total) trans 4294967664 rc 0/0 00000100:00100000:1.0:1789762795.467350:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 122 00000100:00100000:1.0:1789762795.467362:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268701239223 time 268701333554 state 0x2 ref 0 00000100:00100000:0.0:1789762795.469497:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762795.469549:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067338880 00000100:00100000:1.0:1789762795.469559:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067338880 00000100:00100000:1.0:1789762795.469604:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268701239223 time 268701333554 state 0x0 ref 1 00000100:00100000:1.0:1789762795.469619:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268751333554 time 268701333554 state 0x1 ref 1 00000100:00100000:1.0:1789762795.504328:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268751333554 time 268751387913 state 0x0 ref 1 00000100:00100000:1.0:1789762795.504371:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067338880:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762795.504430:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 96468992, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x2e00d 00010000:00010000:1.0:1789762795.504437:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f17524f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.504466:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:1.0:1789762795.504469:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762795.504474:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f17524f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.504486:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f17524f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762795.518290:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967665 is committed 00002000:00010000:1.0:1789762795.518305:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 96468992 -> 97517568 00002000:00010000:1.0:1789762795.518310:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 188429 -> 190477 00010000:00010000:1.0:1789762795.518321:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f17524f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.518331:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f17524f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.518338:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f17524f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.518343:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.518347:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f17524f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.518353:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f17524f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.518379:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f17524f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.518385:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000173b967f/0xdf5a7ac59f17524f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.518455:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067338880:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14104us (48960us total) trans 4294967665 rc 0/0 00000100:00100000:1.0:1789762795.518466:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 123 00000100:00100000:1.0:1789762795.518478:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268751333554 time 268751387913 state 0x2 ref 0 00000100:00100000:0.0:1789762795.520901:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762795.520961:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067339008 00000100:00100000:1.0:1789762795.520970:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067339008 00000100:00100000:1.0:1789762795.521008:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268751333554 time 268751387913 state 0x0 ref 1 00000100:00100000:1.0:1789762795.521019:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268801387913 time 268751387913 state 0x1 ref 1 00000100:00100000:3.0:1789762795.554475:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268801387913 time 268801533846 state 0x0 ref 1 00000100:00100000:3.0:1789762795.554509:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff883800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067339008:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762795.554578:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 97517568, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x2e80d 00010000:00010000:3.0:1789762795.554587:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f175256 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.554631:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:3.0:1789762795.554635:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.554642:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f175256 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.554656:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f175256 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762795.568793:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967666 is committed 00002000:00010000:1.0:1789762795.568851:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 97517568 -> 98566144 00002000:00010000:1.0:1789762795.568887:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 190477 -> 192525 00010000:00010000:1.0:1789762795.568905:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f175256 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.568952:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f175256 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.568970:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f175256 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.568975:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.568979:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f175256 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.568986:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f175256 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.569001:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f175256 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.569023:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f175256 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.569101:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff883800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067339008:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14605us (48198us total) trans 4294967666 rc 0/0 00000100:00100000:1.0:1789762795.569111:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 124 00000100:00100000:1.0:1789762795.569122:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268801387913 time 268801533846 state 0x2 ref 0 00000100:00100000:0.0:1789762795.571960:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762795.571998:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067339136 00000100:00100000:1.0:1789762795.572008:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067339136 00000100:00100000:1.0:1789762795.572054:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268801387913 time 268801533846 state 0x0 ref 1 00000100:00100000:1.0:1789762795.572077:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268851533846 time 268801533846 state 0x1 ref 1 00000100:00100000:2.0:1789762795.604555:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268851533846 time 268851624685 state 0x0 ref 1 00000100:00100000:2.0:1789762795.604572:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067339136:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762795.604609:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 98566144, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x2f00d 00010000:00010000:2.0:1789762795.604616:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca3a9cd6/0xdf5a7ac59f17525d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.604628:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:2.0:1789762795.604632:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762795.604636:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca3a9cd6/0xdf5a7ac59f17525d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.604646:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca3a9cd6/0xdf5a7ac59f17525d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762795.618790:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967667 is committed 00002000:00010000:1.0:1789762795.618812:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 98566144 -> 99614720 00002000:00010000:1.0:1789762795.618824:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 192525 -> 194573 00010000:00010000:1.0:1789762795.618840:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca3a9cd6/0xdf5a7ac59f17525d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.618857:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca3a9cd6/0xdf5a7ac59f17525d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.618867:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca3a9cd6/0xdf5a7ac59f17525d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.618889:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.618896:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca3a9cd6/0xdf5a7ac59f17525d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.618906:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca3a9cd6/0xdf5a7ac59f17525d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.618959:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca3a9cd6/0xdf5a7ac59f17525d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.618966:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca3a9cd6/0xdf5a7ac59f17525d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.619037:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067339136:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14469us (47076us total) trans 4294967667 rc 0/0 00000100:00100000:1.0:1789762795.619047:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 125 00000100:00100000:1.0:1789762795.619056:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268851533846 time 268851624685 state 0x2 ref 0 00000100:00100000:0.0:1789762795.621789:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762795.621906:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067339264 00000100:00100000:1.0:1789762795.621939:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067339264 00000100:00100000:1.0:1789762795.621987:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268851533846 time 268851624685 state 0x0 ref 1 00000100:00100000:1.0:1789762795.622002:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268901624685 time 268851624685 state 0x1 ref 1 00000100:00100000:3.0:1789762795.654732:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268901624685 time 268901794714 state 0x0 ref 1 00000100:00100000:3.0:1789762795.654774:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff883100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067339264:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762795.654842:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 99614720, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x2f80d 00010000:00010000:3.0:1789762795.654851:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f175264 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.654869:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:3.0:1789762795.654886:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.654896:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f175264 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.654969:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f175264 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762795.672730:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967668 is committed 00002000:00010000:1.0:1789762795.672868:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 99614720 -> 100663296 00002000:00010000:1.0:1789762795.672895:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 194573 -> 196621 00010000:00010000:1.0:1789762795.672914:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f175264 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.672960:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f175264 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.672973:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f175264 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.672985:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.672992:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f175264 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.673020:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f175264 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.673062:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f175264 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.673074:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f175264 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.673188:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff883100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067339264:12345-192.168.203.37@tcp:4:dd.500 Request processed in 18436us (51397us total) trans 4294967668 rc 0/0 00000100:00100000:1.0:1789762795.673198:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 126 00000100:00100000:1.0:1789762795.673227:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268901624685 time 268901794714 state 0x2 ref 0 00000100:00100000:0.0:1789762795.676332:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762795.676441:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067339392 00000100:00100000:1.0:1789762795.676451:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067339392 00000100:00100000:1.0:1789762795.676500:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268901624685 time 268901794714 state 0x0 ref 1 00000100:00100000:1.0:1789762795.676514:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268951794714 time 268901794714 state 0x1 ref 1 00000100:00100000:0.0:1789762795.704974:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268951794714 time 268952039917 state 0x0 ref 1 00000100:00100000:0.0:1789762795.705000:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff883b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067339392:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762795.705051:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 100663296, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x3000d 00010000:00010000:0.0:1789762795.705077:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17526b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.705094:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=268s v=(0 0) 00010000:00010000:0.0:1789762795.705099:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762795.705105:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17526b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.705119:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17526b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762795.723451:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 100663296 -> 101711872 00000001:00080000:3.0:1789762795.723460:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967669 is committed 00002000:00010000:1.0:1789762795.723461:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 196621 -> 198669 00010000:00010000:1.0:1789762795.723484:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17526b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.723519:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17526b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.723530:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17526b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.723538:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.723543:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17526b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.723555:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17526b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.723577:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17526b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.723585:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f17526b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.723680:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff883b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067339392:12345-192.168.203.37@tcp:4:dd.500 Request processed in 18684us (47363us total) trans 4294967669 rc 0/0 00000100:00100000:1.0:1789762795.723692:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 127 00000100:00100000:1.0:1789762795.723706:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268951794714 time 268952039917 state 0x2 ref 0 00000100:00100000:1.0:1789762795.726523:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762795.726626:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067339520 00000100:00100000:2.0:1789762795.726642:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067339520 00000100:00100000:2.0:1789762795.726699:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 268951794714 time 268952039917 state 0x0 ref 1 00000100:00100000:2.0:1789762795.726715:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269002039917 time 268952039917 state 0x1 ref 1 00000100:00100000:3.0:1789762795.755103:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269002039917 time 269002170227 state 0x0 ref 1 00000100:00100000:3.0:1789762795.755126:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067339520:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762795.755171:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 101711872, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x3080d 00010000:00010000:3.0:1789762795.755193:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175272 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.755214:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=269s v=(0 0) 00010000:00010000:3.0:1789762795.755219:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.755224:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175272 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.755239:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175272 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762795.776395:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967670 is committed 00002000:00010000:1.0:1789762795.776531:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 101711872 -> 102760448 00002000:00010000:1.0:1789762795.776538:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 198669 -> 200717 00010000:00010000:1.0:1789762795.776553:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175272 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.776566:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175272 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.776576:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175272 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.776583:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.776588:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175272 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.776599:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175272 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.776627:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175272 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.776640:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175272 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.776797:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067339520:12345-192.168.203.37@tcp:4:dd.500 Request processed in 21671us (50269us total) trans 4294967670 rc 0/0 00000100:00100000:1.0:1789762795.776812:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 128 00000100:00100000:1.0:1789762795.776851:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269002039917 time 269002170227 state 0x2 ref 0 00000100:00100000:1.0:1789762795.779523:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762795.779582:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067339648 00000100:00100000:3.0:1789762795.779594:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067339648 00000100:00100000:3.0:1789762795.779646:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269002039917 time 269002170227 state 0x0 ref 1 00000100:00100000:3.0:1789762795.779662:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269052170227 time 269002170227 state 0x1 ref 1 00000100:00100000:3.0:1789762795.805181:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269052170227 time 269052247265 state 0x0 ref 1 00000100:00100000:3.0:1789762795.805204:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067339648:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762795.805268:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 102760448, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x3100d 00010000:00010000:3.0:1789762795.805277:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175279 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.805294:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=269s v=(0 0) 00010000:00010000:3.0:1789762795.805299:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.805305:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175279 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.805319:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175279 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762795.820046:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967671 is committed 00002000:00010000:1.0:1789762795.820061:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 102760448 -> 103809024 00002000:00010000:1.0:1789762795.820070:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 200717 -> 202765 00010000:00010000:1.0:1789762795.820086:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175279 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.820102:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175279 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.820113:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175279 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.820122:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.820128:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175279 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.820139:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175279 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.820162:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175279 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.820173:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089c8afab/0xdf5a7ac59f175279 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.820286:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067339648:12345-192.168.203.37@tcp:4:dd.500 Request processed in 15085us (40759us total) trans 4294967671 rc 0/0 00000100:00100000:1.0:1789762795.820301:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 129 00000100:00100000:1.0:1789762795.820316:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269052170227 time 269052247265 state 0x2 ref 0 00000100:00100000:1.0:1789762795.822465:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762795.822528:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067339776 00000100:00100000:2.0:1789762795.822540:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067339776 00000100:00100000:2.0:1789762795.822616:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269052170227 time 269052247265 state 0x0 ref 1 00000100:00100000:2.0:1789762795.822639:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269102247265 time 269052247265 state 0x1 ref 1 00000100:00100000:1.0:1789762795.855324:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269102247265 time 269102388121 state 0x0 ref 1 00000100:00100000:1.0:1789762795.855362:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067339776:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762795.855416:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 103809024, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x3180d 00010000:00010000:1.0:1789762795.855426:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175280 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.855444:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=269s v=(0 0) 00010000:00010000:1.0:1789762795.855449:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762795.855457:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175280 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.855473:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175280 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762795.876179:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967672 is committed 00002000:00010000:1.0:1789762795.876205:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb size from disk: 103809024 -> 104857600 00002000:00010000:1.0:1789762795.876215:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x5:0x0] updating lvb blocks from disk: 202765 -> 204813 00010000:00010000:1.0:1789762795.876233:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175280 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.876261:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175280 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.876270:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175280 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.876277:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.876281:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175280 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.876290:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175280 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.876309:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175280 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.876315:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175280 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.876409:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067339776:12345-192.168.203.37@tcp:4:dd.500 Request processed in 21052us (53941us total) trans 4294967672 rc 0/0 00000100:00100000:1.0:1789762795.876420:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 130 00000100:00100000:1.0:1789762795.876432:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269102247265 time 269102388121 state 0x2 ref 0 00000100:00100000:0.0:1789762795.878363:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762795.878424:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067340032 00000100:00100000:2.0:1789762795.878435:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067340032 00000100:00100000:2.0:1789762795.878516:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 49 00000100:00080000:1.0:1789762795.878535:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762795.878546:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13384:x1876702067340032:12345-192.168.203.37@tcp:35:dd.500 00000001:00080000:2.0:1789762795.878891:0:6421:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967373 00000100:00100000:2.0:1789762795.879008:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:13384:x1876702067340032:12345-192.168.203.37@tcp:35:dd.500 Request processed in 465us (665us total) trans 4294967374 rc 0/0 00000100:00100000:2.0:1789762795.879016:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 49 00000020:00080000:2.0:1789762795.879020:0:6421:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789762795.902026:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762795.902083:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067340160 00000100:00100000:0.0:1789762795.902090:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067340160 00000100:00100000:0.0:1789762795.902136:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 30 00000100:00100000:0.0:1789762795.902145:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:6823:x1876702067340160:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762795.902156:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00002000:00010000:0.0:1789762795.902169:0:8070:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x6:0x0] updating lvb blocks from disk: 4103 -> 16391 00010000:00010000:0.0:1789762795.902188:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fabe0ddb/0xdf5a7ac59f174e2e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x6:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c61e expref: 8 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.902206:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f174e35 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c625 expref: 7 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.902217:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f174e3c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c62c expref: 6 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.902229:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762795.902291:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:6823:x1876702067340160:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 146us (268us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762795.902299:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 30 00000100:00100000:3.0:1789762795.923744:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762795.923835:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067340416 00000100:00100000:2.0:1789762795.923850:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067340416 00000100:00100000:2.0:1789762795.924007:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 198 00000100:00100000:2.0:1789762795.924042:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff687480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:13400:x1876702067340416:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:2.0:1789762795.924072:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762795.924148:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10600/0xdf5a7ac59f175287 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c680 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.924331:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f17528e lrc: 3/1,0 mode: --/CR res: [0x200000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.924354:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f17528e lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x10:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.924376:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f17528e lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x10:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762795.924411:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x10:0x0]: returning size 104857600 00010000:00010000:2.0:1789762795.924580:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10600/0xdf5a7ac59f175287 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c680 expref: 23 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.924610:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f17528e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x10:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c680 expref: 22 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762795.924625:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff9c37c72bae00) 00010000:00010000:2.0:1789762795.924630:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:2.0:1789762795.924635:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37e1b10200, rc 0) 00000100:00100000:2.0:1789762795.924712:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff687480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:13400:x1876702067340416:12345-192.168.203.37@tcp:101:dd.500 Request processed in 674us (974us total) trans 4294967375 rc 0/0 00000100:00100000:2.0:1789762795.924725:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 198 00000100:00100000:2.0:1789762795.926244:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762795.926331:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067340544 00000100:00100000:0.0:1789762795.926341:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067340544 00000100:00100000:0.0:1789762795.926413:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 128 00000100:00100000:0.0:1789762795.926426:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067340544:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:0.0:1789762795.926450:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762795.926465:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052f30ec7/0xdf5a7ac59f17529c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c687 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762795.926488:0:9037:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 104857600, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x3200d 00010000:00010000:0.0:1789762795.926501:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=269s v=(0 0) 00010000:00010000:0.0:1789762795.926506:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762795.926514:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052f30ec7/0xdf5a7ac59f17529c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c687 expref: 6 pid: 9037 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.926546:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000052f30ec7/0xdf5a7ac59f17529c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c687 expref: 6 pid: 9037 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762795.926558:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d1fa6800, rc 0) 00000100:00100000:0.0:1789762795.926602:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067340544:12345-192.168.203.37@tcp:101:dd.500 Request processed in 179us (377us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762795.926610:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 128 00000100:00100000:1.0:1789762795.929198:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762795.929275:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067340672 00000100:00100000:3.0:1789762795.929283:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067340672 00000100:00100000:3.0:1789762795.929336:0:14171:0:(nrs_tbf.c:1585:nrs_tbf_cli_findadd()) unknown.0 NRS: TBF class insert: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 3 deadline 0 time 269176405887 state 0x0 ref 1 00000100:00100000:3.0:1789762795.929367:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 3 deadline 269276405887 time 269176405887 state 0x1 ref 1 00000100:00100000:3.0:1789762795.929406:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 2 deadline 269276405887 time 269176482644 state 0x0 ref 1 00000100:00100000:3.0:1789762795.929417:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067340672:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762795.929440:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1752a3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.929466:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.929471:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.929478:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1752a3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.929493:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1752a3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.933984:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1752a3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.933998:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1752a3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.934008:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1752a3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.934015:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.934020:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1752a3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.934048:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1752a3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.934067:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1752a3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.934076:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000058de20ee/0xdf5a7ac59f1752a3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.934143:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067340672:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4725us (4948us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762795.934154:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 131 00000100:00100000:1.0:1789762795.934165:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 2 deadline 269276405887 time 269176482644 state 0x2 ref 0 00000100:00100000:1.0:1789762795.936192:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762795.936254:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067340800 00000100:00100000:3.0:1789762795.936262:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067340800 00000100:00100000:3.0:1789762795.936297:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 2 deadline 269276405887 time 269176482644 state 0x0 ref 1 00000100:00100000:3.0:1789762795.936313:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 2 deadline 269276482644 time 269176482644 state 0x1 ref 1 00000100:00100000:3.0:1789762795.936351:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 1 deadline 269276482644 time 269183424117 state 0x0 ref 1 00000100:00100000:3.0:1789762795.936368:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067340800:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762795.936387:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f1752aa lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.936398:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.936401:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762795.936405:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f1752aa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.936428:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f1752aa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.939806:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f1752aa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.939820:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f1752aa lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.939829:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f1752aa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.939853:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762795.939856:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f1752aa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.939863:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f1752aa lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.939895:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f1752aa lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762795.939906:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f1752aa lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762795.940044:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067340800:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3677us (3851us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762795.940065:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 132 00000100:00100000:3.0:1789762795.940075:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 1 deadline 269276482644 time 269183424117 state 0x2 ref 0 00000100:00100000:2.0:1789762795.941805:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762795.941989:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067340928 00000100:00100000:1.0:1789762795.941998:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067340928 00000100:00100000:1.0:1789762795.942031:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 1 deadline 269276482644 time 269183424117 state 0x0 ref 1 00000100:00100000:1.0:1789762795.942042:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 1 deadline 269283424117 time 269183424117 state 0x1 ref 1 00000100:00100000:1.0:1789762795.942069:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269283424117 time 269189146872 state 0x0 ref 1 00000100:00100000:1.0:1789762795.942098:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067340928:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762795.942124:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.942138:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762795.942143:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762795.942148:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.942161:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.945987:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.946001:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.946018:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.946025:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762795.946029:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.946037:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.946053:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762795.946078:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762795.946137:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067340928:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4056us (4333us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762795.946146:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 133 00000100:00100000:1.0:1789762795.946156:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269283424117 time 269189146872 state 0x2 ref 0 00000100:00100000:1.0:1789762795.947998:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762795.948052:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067341056 00000100:00100000:3.0:1789762795.948060:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067341056 00000100:00100000:3.0:1789762795.948091:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269283424117 time 269189146872 state 0x0 ref 1 00000100:00100000:3.0:1789762795.948102:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269289146872 time 269189146872 state 0x1 ref 1 00000100:00100000:2.0:1789762796.032182:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074654592:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762796.032348:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762796.032403:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074654592 00000100:00100000:3.0:1789762796.032418:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074654592 00000100:00100000:3.0:1789762796.032505:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00100000:3.0:1789762796.032523:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074654592:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789762796.032569:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789762796.032652:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074654592:12345-0@lo:41:osp-pre-0-1.0 Request processed in 135us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762796.032662:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00100000:2.0:1789762796.032724:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074654592:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762796.042640:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269289146872 time 269289704414 state 0x0 ref 1 00000100:00100000:1.0:1789762796.042665:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067341056:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762796.042732:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.042760:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762796.042770:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762796.042782:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.042809:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.047038:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.047057:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.047070:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.047081:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762796.047087:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.047099:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.047122:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.047131:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1752b8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762796.047225:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067341056:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4546us (99207us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762796.047260:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 134 00000100:00100000:2.0:1789762796.047277:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269289146872 time 269289704414 state 0x2 ref 0 00000001:00080000:2.0:1789762796.048467:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967370 is committed 00000001:00080000:2.0:1789762796.048511:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967375 is committed 00010000:00010000:1.0:1789762796.048522:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048536:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048545:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048553:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762796.048562:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.048564:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f77 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048577:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.048588:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f77 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048598:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee100) 00010000:00010000:1.0:1789762796.048605:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762796.048608:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f77 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048624:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.048629:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762796.048652:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10a00/0xdf5a7ac59f174ef2 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.048669:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f77 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048675:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174ef9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048697:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174ef9 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048723:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174ef9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.048728:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f77 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048730:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762796.048739:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174ef9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048759:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174ef9 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.048760:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:3.0:1789762796.048772:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789762796.048781:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f77 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048785:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:3.0:1789762796.048791:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b11400/0xdf5a7ac59f174f77 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048792:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762796.048800:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174ef9 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048812:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d1fa6a00/0xdf5a7ac59f174ef9 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.048824:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174fa8 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048832:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174f54 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.048837:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174fa8 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048847:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174f54 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.048852:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174fa8 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.048896:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762796.048902:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174f54 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.048904:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174fa8 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.048964:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762796.048970:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174f54 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.049006:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174f54 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.049012:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174fa8 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.049043:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:3.0:1789762796.049045:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x4009f790 (ffff9c37c72ba300) 00010000:00010000:1.0:1789762796.049058:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789762796.049058:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762796.049068:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174f54 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.049078:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174fa8 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.049089:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f174f54 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.049093:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff935a00/0xdf5a7ac59f174fa8 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762796.049341:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762796.049392:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067341184 00000100:00100000:1.0:1789762796.049400:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067341184 00000100:00100000:1.0:1789762796.049444:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269289146872 time 269289704414 state 0x0 ref 1 00000100:00100000:1.0:1789762796.049459:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269389704414 time 269289704414 state 0x1 ref 1 00000100:00100000:2.0:1789762796.142755:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269389704414 time 269389816142 state 0x0 ref 1 00000100:00100000:2.0:1789762796.142785:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067341184:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762796.142827:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1752bf lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.142844:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762796.142849:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762796.142855:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1752bf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.142868:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1752bf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.146734:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1752bf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.146747:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1752bf lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.146770:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1752bf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.146777:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762796.146782:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1752bf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.146790:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1752bf lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.146806:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1752bf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.146813:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1752bf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762796.146892:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067341184:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4118us (97551us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762796.146901:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 135 00000100:00100000:1.0:1789762796.146911:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269389704414 time 269389816142 state 0x2 ref 0 00000100:00100000:3.0:1789762796.149410:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762796.149459:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067341312 00000100:00100000:1.0:1789762796.149469:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067341312 00000100:00100000:1.0:1789762796.149512:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269389704414 time 269389816142 state 0x0 ref 1 00000100:00100000:1.0:1789762796.149538:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269489816142 time 269389816142 state 0x1 ref 1 00010000:00010000:1.0:1789762796.160128:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762796.242807:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269489816142 time 269489871909 state 0x0 ref 1 00000100:00100000:1.0:1789762796.242831:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff685500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067341312:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762796.242868:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000afeac79e/0xdf5a7ac59f1752c6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.242899:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762796.242904:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762796.242911:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afeac79e/0xdf5a7ac59f1752c6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.242957:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afeac79e/0xdf5a7ac59f1752c6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.245574:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000afeac79e/0xdf5a7ac59f1752c6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.245584:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afeac79e/0xdf5a7ac59f1752c6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.245590:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afeac79e/0xdf5a7ac59f1752c6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.245606:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d926b000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762796.245610:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afeac79e/0xdf5a7ac59f1752c6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.245636:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afeac79e/0xdf5a7ac59f1752c6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.245656:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afeac79e/0xdf5a7ac59f1752c6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.245663:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afeac79e/0xdf5a7ac59f1752c6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762796.245731:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff685500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067341312:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2909us (96324us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762796.245738:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 136 00000100:00100000:1.0:1789762796.245746:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269489816142 time 269489871909 state 0x2 ref 0 00000100:00100000:2.0:1789762796.247782:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762796.247821:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067341440 00000100:00100000:1.0:1789762796.247829:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067341440 00000100:00100000:1.0:1789762796.247863:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269489816142 time 269489871909 state 0x0 ref 1 00000100:00100000:1.0:1789762796.247889:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269589871909 time 269489871909 state 0x1 ref 1 00000100:00100000:3.0:1789762796.343089:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269589871909 time 269590155066 state 0x0 ref 1 00000100:00100000:3.0:1789762796.343114:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067341440:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762796.343151:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752cd lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.343165:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762796.343168:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762796.343173:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752cd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.343184:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752cd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.346521:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752cd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.346551:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752cd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.346558:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752cd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.346575:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762796.346580:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752cd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.346587:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752cd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.346604:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752cd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.346613:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752cd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762796.346683:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067341440:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3578us (98901us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762796.346691:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 137 00000100:00100000:3.0:1789762796.346698:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269589871909 time 269590155066 state 0x2 ref 0 00000100:00100000:2.0:1789762796.348826:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762796.348884:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067341568 00000100:00100000:1.0:1789762796.348902:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067341568 00000100:00100000:1.0:1789762796.348969:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269589871909 time 269590155066 state 0x0 ref 1 00000100:00100000:1.0:1789762796.348985:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269690155066 time 269590155066 state 0x1 ref 1 00000100:00100000:3.0:1789762796.443202:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269690155066 time 269690260344 state 0x0 ref 1 00000100:00100000:3.0:1789762796.443237:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067341568:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762796.443283:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752d4 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.443322:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762796.443326:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762796.443333:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752d4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.443348:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752d4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.446901:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752d4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.446935:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752d4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.446947:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752d4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.446955:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762796.446961:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752d4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.446972:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752d4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.446991:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752d4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.447014:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752d4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762796.447079:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067341568:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3857us (98252us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762796.447090:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 138 00000100:00100000:3.0:1789762796.447102:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269690155066 time 269690260344 state 0x2 ref 0 00000100:00100000:2.0:1789762796.449038:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762796.449098:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067341696 00000100:00100000:1.0:1789762796.449106:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067341696 00000100:00100000:1.0:1789762796.449146:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269690155066 time 269690260344 state 0x0 ref 1 00000100:00100000:1.0:1789762796.449158:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269790260344 time 269690260344 state 0x1 ref 1 00000100:00100000:1.0:1789762796.543290:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269790260344 time 269790354078 state 0x0 ref 1 00000100:00100000:1.0:1789762796.543314:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067341696:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762796.543351:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f1752db lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.543365:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762796.543377:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762796.543382:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f1752db lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.543393:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f1752db lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.546585:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f1752db lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.546621:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f1752db lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.546649:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f1752db lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.546658:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f0400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762796.546663:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f1752db lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.546674:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f1752db lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.546696:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f1752db lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.546705:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f1752db lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762796.546790:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067341696:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3481us (97749us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762796.546801:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 139 00000100:00100000:2.0:1789762796.546810:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269790260344 time 269790354078 state 0x2 ref 0 00000100:00100000:2.0:1789762796.548788:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762796.548835:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067341824 00000100:00100000:3.0:1789762796.548843:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067341824 00000100:00100000:3.0:1789762796.548897:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269790260344 time 269790354078 state 0x0 ref 1 00000100:00100000:3.0:1789762796.548913:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269890354078 time 269790354078 state 0x1 ref 1 00000100:00100000:2.0:1789762796.643436:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269890354078 time 269890482639 state 0x0 ref 1 00000100:00100000:2.0:1789762796.643475:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067341824:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762796.643517:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000052b012fc/0xdf5a7ac59f1752e2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.643534:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762796.643538:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762796.643544:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052b012fc/0xdf5a7ac59f1752e2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762796.643561:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052b012fc/0xdf5a7ac59f1752e2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.647129:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000052b012fc/0xdf5a7ac59f1752e2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.647158:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000052b012fc/0xdf5a7ac59f1752e2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.647167:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000052b012fc/0xdf5a7ac59f1752e2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.647175:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762796.647179:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000052b012fc/0xdf5a7ac59f1752e2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.647188:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000052b012fc/0xdf5a7ac59f1752e2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.647205:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000052b012fc/0xdf5a7ac59f1752e2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.647212:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052b012fc/0xdf5a7ac59f1752e2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762796.647291:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067341824:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3829us (98510us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762796.647304:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 140 00000100:00100000:1.0:1789762796.647317:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269890354078 time 269890482639 state 0x2 ref 0 00000100:00100000:1.0:1789762796.649484:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762796.649548:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067341952 00000100:00100000:3.0:1789762796.649559:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067341952 00000100:00100000:3.0:1789762796.649625:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269890354078 time 269890482639 state 0x0 ref 1 00000100:00100000:3.0:1789762796.649645:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269990482639 time 269890482639 state 0x1 ref 1 00000100:00100000:3.0:1789762796.743477:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269990482639 time 269990536436 state 0x0 ref 1 00000100:00100000:3.0:1789762796.743506:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067341952:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762796.743578:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752e9 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.743597:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762796.743602:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762796.743608:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752e9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.743622:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752e9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.746891:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752e9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.746907:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752e9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.746950:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752e9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.746960:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762796.746966:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752e9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.746979:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752e9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.747022:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752e9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.747033:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752e9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762796.747128:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067341952:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3628us (97649us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762796.747155:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 141 00000100:00100000:3.0:1789762796.747170:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269990482639 time 269990536436 state 0x2 ref 0 00000100:00100000:1.0:1789762796.749286:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762796.749355:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067342080 00000100:00100000:2.0:1789762796.749370:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067342080 00000100:00100000:2.0:1789762796.749487:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 269990482639 time 269990536436 state 0x0 ref 1 00000100:00100000:2.0:1789762796.749504:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270090536436 time 269990536436 state 0x1 ref 1 00000100:00100000:1.0:1789762796.843592:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270090536436 time 270090648453 state 0x0 ref 1 00000100:00100000:1.0:1789762796.843631:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067342080:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762796.843683:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.843702:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762796.843706:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762796.843712:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.843729:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.847684:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.847695:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.847703:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.847709:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762796.847713:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.847735:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.847753:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.847758:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762796.847820:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067342080:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4204us (98535us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762796.847828:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 142 00000100:00100000:1.0:1789762796.847836:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270090536436 time 270090648453 state 0x2 ref 0 00000100:00100000:1.0:1789762796.849947:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762796.850032:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067342208 00000100:00100000:2.0:1789762796.850049:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067342208 00000100:00100000:2.0:1789762796.850123:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270090536436 time 270090648453 state 0x0 ref 1 00000100:00100000:2.0:1789762796.850144:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270190648453 time 270090648453 state 0x1 ref 1 00000100:00100000:1.0:1789762796.943811:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270190648453 time 270190868584 state 0x0 ref 1 00000100:00100000:1.0:1789762796.943846:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067342208:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762796.943963:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.943987:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762796.943993:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762796.944001:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762796.944042:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.947725:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.947742:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.947749:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.947754:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762796.947758:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.947765:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.947789:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762796.947795:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f1752f7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762796.947887:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067342208:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4054us (97936us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762796.947897:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 143 00000100:00100000:3.0:1789762796.947908:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270190648453 time 270190868584 state 0x2 ref 0 00000100:00100000:2.0:1789762796.949861:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762796.949908:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067342336 00000100:00100000:3.0:1789762796.949951:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067342336 00000100:00100000:3.0:1789762796.949995:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270190648453 time 270190868584 state 0x0 ref 1 00000100:00100000:3.0:1789762796.950024:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270290868584 time 270190868584 state 0x1 ref 1 00000100:00100000:1.0:1789762796.999805:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762796.999844:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762796.999893:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067342464 00000100:00100000:1.0:1789762796.999904:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762796.999910:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067342464 00000100:00100000:2.0:1789762797.000002:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067342592 00000100:00100000:1.0:1789762797.000005:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762797.000014:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067342592 00000100:00100000:0.0:1789762797.000018:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067342720 00000100:00100000:0.0:1789762797.000028:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067342720 00000100:00100000:3.0:1789762797.000036:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 331 00000100:00100000:1.0:1789762797.000066:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762797.000067:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067342464:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762797.000095:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067343104 00000100:00100000:2.0:1789762797.000101:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067343104 00000100:00100000:0.0:1789762797.000106:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 348 00000100:00100000:0.0:1789762797.000121:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2397:x1876702067342592:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762797.000139:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762797.000147:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 349 00000100:00100000:2.0:1789762797.000169:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067342720:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762797.000171:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067342464:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 111us (363us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762797.000183:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2397:x1876702067342592:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 60us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762797.000200:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 348 00000100:00100000:3.0:1789762797.000200:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 331 00000100:00100000:0.0:1789762797.000228:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067342976 00000100:00100000:2.0:1789762797.000241:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067342720:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 83us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762797.000242:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067342976 00000100:00100000:2.0:1789762797.000261:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 349 00000100:00100000:1.0:1789762797.000299:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067342848 00000100:00100000:1.0:1789762797.000305:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067342848 00000100:00100000:0.0:1789762797.000312:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 350 00000100:00100000:2.0:1789762797.000323:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 351 00000100:00100000:0.0:1789762797.000335:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067343104:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762797.000337:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067342976:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762797.000358:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 135 00000100:00100000:1.0:1789762797.000370:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067342848:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762797.000380:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067342976:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 44us (240us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762797.000386:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 351 00000100:00100000:0.0:1789762797.000401:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067343104:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 71us (344us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762797.000414:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067342848:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 46us (412us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762797.000419:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 350 00000100:00100000:1.0:1789762797.000429:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 135 00000100:00080000:0.0:1789762797.000489:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762797.000498:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762797.000508:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762797.043910:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270290868584 time 270290968041 state 0x0 ref 1 00000100:00100000:3.0:1789762797.043976:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067342336:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:0.0:1789762797.044006:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762797.044022:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752fe lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.044043:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762797.044048:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762797.044056:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752fe lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.044074:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752fe lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.047466:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752fe lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.047488:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752fe lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.047497:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752fe lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.047523:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762797.047530:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752fe lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.047541:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752fe lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.047562:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752fe lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.047569:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a11c428/0xdf5a7ac59f1752fe lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762797.047649:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067342336:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3695us (97786us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762797.047659:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 144 00000100:00100000:1.0:1789762797.047670:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270290868584 time 270290968041 state 0x2 ref 0 00000100:00100000:2.0:1789762797.049559:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762797.049596:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067343232 00000100:00100000:1.0:1789762797.049604:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067343232 00000100:00100000:1.0:1789762797.049646:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270290868584 time 270290968041 state 0x0 ref 1 00000100:00100000:1.0:1789762797.049657:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270390968041 time 270290968041 state 0x1 ref 1 00000020:00080000:3.0:1789762797.064910:0:7221:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 6 batchid updated 00000004:00080000:3.0:1789762797.064948:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 10 committed 00000020:00080000:0.0:1789762797.064977:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 6 00000004:00080000:3.0:1789762797.064982:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 10 committed 00000004:00080000:0.0:1789762797.065035:0:7233:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37ff9c6180 with version 14 00000020:00080000:0.0:1789762797.065061:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 5 cancel update log [0x200000401:0x2:0x0].5: rc = 0 00000100:00100000:3.0:1789762797.065072:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8aaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074654720:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1789762797.065099:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074654848:0@lo:6:osp-syn-1-1.0 00000100:00100000:3.0:1789762797.065139:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762797.065174:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762797.065193:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074654720 00000100:00100000:2.0:1789762797.065202:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074654720 00000100:00080000:3.0:1789762797.065252:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762797.065274:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074654848 00000100:00100000:1.0:1789762797.065281:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074654848 00000100:00100000:2.0:1789762797.065303:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 129 00000100:00100000:2.0:1789762797.065320:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff686680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074654720:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:2.0:1789762797.065347:0:9038:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:6 count 1 00000100:00100000:1.0:1789762797.065349:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 130 00000100:00100000:1.0:1789762797.065367:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074654848:12345-0@lo:6:osp-syn-1-1.0 00002000:00010000:2.0:1789762797.065379:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x6:0x0] initial LVB size: 8388608, mtime: 0x6aad9ce5, atime: 0x0, ctime: 0x6aad9ce5, blocks: 0x4007 00002000:00080000:1.0:1789762797.065387:0:8054:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:6 count 1 00010000:00010000:2.0:1789762797.065388:0:9038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004103f0bc/0xdf5a7ac59f175305 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.065421:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x6:0x0].0x0 since=270s v=(0 0) 00002000:00010000:1.0:1789762797.065423:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x6:0x0] initial LVB size: 8388608, mtime: 0x6aad9ce5, atime: 0x0, ctime: 0x6aad9ce5, blocks: 0x4007 00010000:00010000:2.0:1789762797.065433:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762797.065435:0:8054:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000007b7c1fe/0xdf5a7ac59f17530c lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.065453:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004103f0bc/0xdf5a7ac59f175305 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.065456:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x6:0x0].0x0 since=270s v=(0 0) 00010000:00010000:1.0:1789762797.065469:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762797.065486:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000007b7c1fe/0xdf5a7ac59f17530c lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.065494:0:9038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004103f0bc/0xdf5a7ac59f175305 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.065501:0:8054:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000007b7c1fe/0xdf5a7ac59f17530c lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.065521:0:9038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004103f0bc/0xdf5a7ac59f175305 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.065523:0:8054:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000007b7c1fe/0xdf5a7ac59f17530c lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.065568:0:9038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004103f0bc/0xdf5a7ac59f175305 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.065569:0:8054:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000007b7c1fe/0xdf5a7ac59f17530c lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.065627:0:8054:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1789762797.065630:0:9038:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1789762797.065644:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1789762797.065656:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000007b7c1fe/0xdf5a7ac59f17530c lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.065667:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff671600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762797.065674:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000007b7c1fe/0xdf5a7ac59f17530c lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.065696:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000007b7c1fe/0xdf5a7ac59f17530c lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.065715:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000007b7c1fe/0xdf5a7ac59f17530c lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.065722:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000007b7c1fe/0xdf5a7ac59f17530c lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.065744:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762797.065748:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004103f0bc/0xdf5a7ac59f175305 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.065755:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9ba8c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762797.065761:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004103f0bc/0xdf5a7ac59f175305 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.065768:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004103f0bc/0xdf5a7ac59f175305 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.065779:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004103f0bc/0xdf5a7ac59f175305 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.065785:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004103f0bc/0xdf5a7ac59f175305 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1789762797.065817:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 5 cancel update log [0x240000400:0x2:0x0].5: rc = 0 00010000:00010000:3.0:1789762797.065831:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762797.065836:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762797.065839:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762797.065848:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1789762797.065890:0:7233:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37ff9c6840 with version 15 00000020:00080000:0.0:1789762797.065894:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 6 cancel update log [0x200000401:0x2:0x0].6: rc = 0 00000020:00080000:0.0:1789762797.066443:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 6 cancel update log [0x240000400:0x2:0x0].6: rc = 0 00000020:00080000:0.0:1789762797.066455:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 6 00000004:00080000:0.0:1789762797.066466:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 14 rpc_version 14 00000020:00100000:0.0:1789762797.066509:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762797.066518:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37ff880700 x1876702074655104/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762797.066538:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff880700 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074655104:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762797.066605:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762797.066652:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074655104 00000100:00100000:0.0:1789762797.066655:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff9c6f00 going to sleep for 11 seconds 00000100:00100000:3.0:1789762797.066661:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074655104 00000100:00100000:3.0:1789762797.066724:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 233 00000100:00100000:3.0:1789762797.066737:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:7224:x1876702074655104:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762797.066920:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff686680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074654720:12345-0@lo:6:osp-syn-0-1.0 Request processed in 1604us (1777us total) trans 4294967673 rc 0/0 00000100:00100000:2.0:1789762797.066949:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 129 00000004:00080000:0.0:1789762797.067011:0:3427:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37ff8aaa00/2, rc 0, transno 4294967673 00000100:00100000:0.0:1789762797.067018:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8aaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074654720:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1789762797.067065:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074654848:12345-0@lo:6:osp-syn-1-1.0 Request processed in 1701us (1894us total) trans 4294967363 rc 0/0 00000100:00100000:1.0:1789762797.067086:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 130 00000100:00100000:2.0:1789762797.067108:0:3424:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294967362 gen 2 00000004:00080000:2.0:1789762797.067115:0:3424:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37f7f5aa00, transno 4294967362 00000100:00080000:1.0:1789762797.067126:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762797.067132:0:3424:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9c37c96b4700 x1876702074654848/t4294967363(4294967363) o6->lustre-OST0001-osc-MDT0001@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789762813 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-1.0' uid:0 gid:0 projid:4294967295 00000004:00080000:2.0:1789762797.067141:0:3424:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37c96b4700/2, rc 0, transno 4294967363 00000100:00100000:2.0:1789762797.067151:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074654848:0@lo:6:osp-syn-1-1.0 00000100:00100000:3.0:1789762797.067247:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074655104:12345-0@lo:1000:osp_up0-1.0 Request processed in 515us (644us total) trans 4294967376 rc 0/0 00000100:00100000:3.0:1789762797.067254:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 233 00000100:00100000:2.0:1789762797.067273:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff880700 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074655104:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1789762797.067289:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 15 rpc_version 15 00000020:00100000:2.0:1789762797.067303:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1789762797.067308:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37ff6bb800 x1876702074655360/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762797.067320:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6bb800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074655360:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762797.067350:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762797.067369:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f61c89c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762797.067382:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074655360 00000100:00100000:3.0:1789762797.067387:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074655360 00000100:00100000:3.0:1789762797.067411:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 234 00000100:00100000:3.0:1789762797.067419:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074655360:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1789762797.067575:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7224:x1876702074655360:12345-0@lo:1000:osp_up0-1.0 Request processed in 157us (224us total) trans 4294967377 rc 0/0 00000100:00100000:3.0:1789762797.067583:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 234 00000100:00100000:2.0:1789762797.067615:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6bb800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074655360:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1789762797.144003:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270390968041 time 270391069374 state 0x0 ref 1 00000100:00100000:3.0:1789762797.144029:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067343232:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762797.144066:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f175313 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.144079:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762797.144094:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762797.144099:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f175313 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.144111:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f175313 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.146961:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f175313 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.146970:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f175313 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.146977:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f175313 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.146983:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762797.146988:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f175313 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.146994:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f175313 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.147008:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f175313 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.147014:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b806bacc/0xdf5a7ac59f175313 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762797.147079:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067343232:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3061us (97522us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762797.147086:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 145 00000100:00100000:3.0:1789762797.147094:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270390968041 time 270391069374 state 0x2 ref 0 00000100:00100000:0.0:1789762797.149025:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762797.149062:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067343360 00000100:00100000:3.0:1789762797.149071:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067343360 00000100:00100000:3.0:1789762797.149122:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270390968041 time 270391069374 state 0x0 ref 1 00000100:00100000:3.0:1789762797.149133:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270491069374 time 270391069374 state 0x1 ref 1 00010000:00010000:1.0:1789762797.184064:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762797.244113:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270491069374 time 270491179034 state 0x0 ref 1 00000100:00100000:2.0:1789762797.244136:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067343360:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762797.244170:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000864ea2c2/0xdf5a7ac59f17531a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.244184:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762797.244187:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762797.244192:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000864ea2c2/0xdf5a7ac59f17531a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.244203:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000864ea2c2/0xdf5a7ac59f17531a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.247251:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000864ea2c2/0xdf5a7ac59f17531a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.247268:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000864ea2c2/0xdf5a7ac59f17531a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.247275:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000864ea2c2/0xdf5a7ac59f17531a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.247280:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d108e200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762797.247284:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000864ea2c2/0xdf5a7ac59f17531a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.247291:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000864ea2c2/0xdf5a7ac59f17531a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.247309:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000864ea2c2/0xdf5a7ac59f17531a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.247314:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000864ea2c2/0xdf5a7ac59f17531a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762797.247409:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067343360:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3278us (98382us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762797.247430:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 146 00000100:00100000:1.0:1789762797.247441:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270491069374 time 270491179034 state 0x2 ref 0 00000100:00100000:0.0:1789762797.249333:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762797.249384:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067343488 00000100:00100000:1.0:1789762797.249396:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067343488 00000100:00100000:1.0:1789762797.249450:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270491069374 time 270491179034 state 0x0 ref 1 00000100:00100000:1.0:1789762797.249462:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270591179034 time 270491179034 state 0x1 ref 1 00000100:00100000:2.0:1789762797.344222:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270591179034 time 270591270816 state 0x0 ref 1 00000100:00100000:2.0:1789762797.344251:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067343488:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762797.344295:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f175321 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.344309:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762797.344312:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762797.344317:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f175321 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.344338:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f175321 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.346985:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f175321 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.346995:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f175321 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.347002:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f175321 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.347007:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d108ec00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762797.347012:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f175321 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.347018:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f175321 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.347044:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f175321 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.347050:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f175321 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762797.347109:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067343488:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2866us (97777us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762797.347116:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 147 00000100:00100000:2.0:1789762797.347123:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270591179034 time 270591270816 state 0x2 ref 0 00000100:00100000:2.0:1789762797.349035:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762797.349066:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067343616 00000100:00100000:3.0:1789762797.349074:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067343616 00000100:00100000:3.0:1789762797.349110:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270591179034 time 270591270816 state 0x0 ref 1 00000100:00100000:3.0:1789762797.349120:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270691270816 time 270591270816 state 0x1 ref 1 00000100:00100000:1.0:1789762797.444346:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270691270816 time 270691385753 state 0x0 ref 1 00000100:00100000:1.0:1789762797.444388:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067343616:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762797.444446:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175328 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.444469:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762797.444475:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762797.444482:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175328 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.444498:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175328 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.447996:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175328 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.448039:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175328 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.448052:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175328 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.448059:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f604ce00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762797.448064:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175328 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.448073:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175328 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.448107:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175328 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.448113:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175328 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762797.448175:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067343616:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3805us (99139us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762797.448182:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 148 00000100:00100000:1.0:1789762797.448192:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270691270816 time 270691385753 state 0x2 ref 0 00000100:00100000:2.0:1789762797.450586:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762797.450627:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067343744 00000100:00100000:1.0:1789762797.450649:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067343744 00000100:00100000:1.0:1789762797.450717:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270691270816 time 270691385753 state 0x0 ref 1 00000100:00100000:1.0:1789762797.450732:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270791385753 time 270691385753 state 0x1 ref 1 00000100:00100000:2.0:1789762797.544433:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270791385753 time 270791497633 state 0x0 ref 1 00000100:00100000:2.0:1789762797.544471:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067343744:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762797.544519:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f17532f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.544533:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762797.544537:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762797.544542:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f17532f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.544569:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f17532f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.548412:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f17532f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.548437:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f17532f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.548448:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f17532f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.548457:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d108ec00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762797.548463:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f17532f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.548474:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f17532f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.548498:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f17532f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.548506:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000172f09bd/0xdf5a7ac59f17532f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762797.548636:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067343744:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4170us (98051us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762797.548654:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 149 00000100:00100000:1.0:1789762797.548672:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270791385753 time 270791497633 state 0x2 ref 0 00000100:00100000:2.0:1789762797.550963:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762797.551008:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067343872 00000100:00100000:1.0:1789762797.551019:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067343872 00000100:00100000:1.0:1789762797.551073:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270791385753 time 270791497633 state 0x0 ref 1 00000100:00100000:1.0:1789762797.551087:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270891497633 time 270791497633 state 0x1 ref 1 00000100:00100000:1.0:1789762797.644509:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270891497633 time 270891572441 state 0x0 ref 1 00000100:00100000:1.0:1789762797.644538:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067343872:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762797.644582:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175336 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.644600:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762797.644604:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762797.644618:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175336 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.644635:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175336 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.647905:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175336 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.647937:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175336 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.647946:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175336 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.647953:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f604ce00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762797.647958:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175336 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.647967:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175336 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.648003:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175336 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.648011:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175336 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762797.648100:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067343872:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3571us (97140us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762797.648112:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 150 00000100:00100000:1.0:1789762797.648132:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270891497633 time 270891572441 state 0x2 ref 0 00000100:00100000:3.0:1789762797.649910:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762797.649989:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067344000 00000100:00100000:1.0:1789762797.650000:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067344000 00000100:00100000:1.0:1789762797.650049:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270891497633 time 270891572441 state 0x0 ref 1 00000100:00100000:1.0:1789762797.650063:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270991572441 time 270891572441 state 0x1 ref 1 00000100:00100000:3.0:1789762797.744619:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270991572441 time 270991679546 state 0x0 ref 1 00000100:00100000:3.0:1789762797.744652:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067344000:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762797.744693:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f17533d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.744723:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762797.744726:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762797.744731:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f17533d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762797.744743:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f17533d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.748441:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f17533d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.748457:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f17533d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.748464:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f17533d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.748470:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762797.748474:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f17533d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.748481:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f17533d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.748510:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f17533d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.748535:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004aa429ed/0xdf5a7ac59f17533d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762797.748627:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067344000:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3987us (98733us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762797.748636:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 151 00000100:00100000:1.0:1789762797.748646:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270991572441 time 270991679546 state 0x2 ref 0 00000100:00100000:2.0:1789762797.750974:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762797.751024:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067344128 00000100:00100000:1.0:1789762797.751035:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067344128 00000100:00100000:1.0:1789762797.751100:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 270991572441 time 270991679546 state 0x0 ref 1 00000100:00100000:1.0:1789762797.751118:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271091679546 time 270991679546 state 0x1 ref 1 00000100:00100000:1.0:1789762797.844729:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271091679546 time 271091780361 state 0x0 ref 1 00000100:00100000:1.0:1789762797.844770:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067344128:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762797.844829:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175344 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.844852:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762797.844873:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762797.844883:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175344 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.844963:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175344 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.848394:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175344 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.848409:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175344 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.848420:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175344 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.848429:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f604ce00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762797.848436:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175344 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.848448:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175344 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.848471:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175344 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762797.848480:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d91441ae/0xdf5a7ac59f175344 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762797.848569:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067344128:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3817us (97595us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762797.848581:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 152 00000100:00100000:1.0:1789762797.848594:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271091679546 time 271091780361 state 0x2 ref 0 00000100:00100000:0.0:1789762797.850608:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762797.850657:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067344256 00000100:00100000:1.0:1789762797.850669:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067344256 00000100:00100000:1.0:1789762797.850721:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271091679546 time 271091780361 state 0x0 ref 1 00000100:00100000:1.0:1789762797.850739:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271191780361 time 271091780361 state 0x1 ref 1 00000100:00100000:2.0:1789762797.944836:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271191780361 time 271191901803 state 0x0 ref 1 00000100:00100000:2.0:1789762797.944884:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067344256:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762797.944954:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000013330ffd/0xdf5a7ac59f17534b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.944971:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762797.944975:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762797.944981:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000013330ffd/0xdf5a7ac59f17534b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.944995:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000013330ffd/0xdf5a7ac59f17534b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.948085:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000013330ffd/0xdf5a7ac59f17534b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.948095:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000013330ffd/0xdf5a7ac59f17534b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.948115:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000013330ffd/0xdf5a7ac59f17534b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.948120:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d98d1600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762797.948123:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000013330ffd/0xdf5a7ac59f17534b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.948129:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000013330ffd/0xdf5a7ac59f17534b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.948142:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000013330ffd/0xdf5a7ac59f17534b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762797.948148:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000013330ffd/0xdf5a7ac59f17534b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762797.948198:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067344256:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3346us (97596us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762797.948205:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 153 00000100:00100000:2.0:1789762797.948212:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271191780361 time 271191901803 state 0x2 ref 0 00000100:00100000:0.0:1789762797.950118:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762797.950174:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067344384 00000100:00100000:1.0:1789762797.950188:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067344384 00000100:00100000:1.0:1789762797.950254:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271191780361 time 271191901803 state 0x0 ref 1 00000100:00100000:1.0:1789762797.950268:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271291901803 time 271191901803 state 0x1 ref 1 00000100:00100000:3.0:1789762798.044955:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271291901803 time 271292021160 state 0x0 ref 1 00000100:00100000:3.0:1789762798.044980:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067344384:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762798.045017:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175352 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.045030:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762798.045034:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762798.045038:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175352 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.045049:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175352 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.047743:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175352 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.047762:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175352 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.047770:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175352 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.047779:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d108e000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762798.047783:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175352 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.047794:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175352 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.047814:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175352 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.047823:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175352 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762798.047898:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067344384:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2929us (97780us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762798.047906:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 154 00000100:00100000:3.0:1789762798.047913:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271291901803 time 271292021160 state 0x2 ref 0 00000100:00100000:1.0:1789762798.049753:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762798.049830:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067344512 00000100:00100000:2.0:1789762798.049841:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067344512 00000100:00100000:2.0:1789762798.049891:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271291901803 time 271292021160 state 0x0 ref 1 00000100:00100000:2.0:1789762798.049913:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271392021160 time 271292021160 state 0x1 ref 1 00000100:00100000:1.0:1789762798.080133:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8ab100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074655488:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762798.080165:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074655872:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762798.080226:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762798.080260:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074655616:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762798.080327:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762798.080367:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074656000:0@lo:400:kworker.0 00000100:00100000:2.0:1789762798.080393:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074656128:0@lo:400:kworker.0 00000100:00100000:0.0:1789762798.080404:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074655488 00000100:00100000:0.0:1789762798.080413:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074655488 00000100:00100000:1.0:1789762798.080426:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762798.080461:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762798.080496:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074655616 00000100:00100000:0.0:1789762798.080501:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074655616 00000100:00100000:1.0:1789762798.080535:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074656512:0@lo:400:kworker.0 00000100:00100000:2.0:1789762798.080554:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074656640:0@lo:400:kworker.0 00000100:00100000:0.0:1789762798.080564:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:0.0:1789762798.080593:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074655488:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762798.080596:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074656000 00000100:00100000:1.0:1789762798.080615:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762798.080616:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074656000 00000100:00100000:2.0:1789762798.080626:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:0.0:1789762798.080648:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3746562048 avail 3744464896 00000100:00100000:1.0:1789762798.080666:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9fff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074657024:0@lo:400:kworker.0 00000100:00100000:2.0:1789762798.080674:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074657152:0@lo:400:kworker.0 00000100:00100000:3.0:1789762798.080684:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:3.0:1789762798.080699:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074656000:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.080739:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762798.080740:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074655488:12345-0@lo:13:osp-pre-0-1.0 Request processed in 149us (512us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762798.080740:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762798.080761:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00100000:3.0:1789762798.080774:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074656000:12345-0@lo:400:kworker.0 Request processed in 78us (349us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762798.080804:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 147 00000100:00100000:2.0:1789762798.080805:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:3.0:1789762798.080805:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00100000:1.0:1789762798.080817:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8ab100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074655488:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762798.080828:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074655616:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762798.080832:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074655744:0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789762798.080855:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3851419648 avail 3849322496 00000100:00100000:1.0:1789762798.080880:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074656000:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.080906:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1789762798.080964:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074656384:0@lo:400:kworker.0 00000100:00100000:2.0:1789762798.080966:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762798.080975:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074655616:12345-0@lo:13:osp-pre-1-1.0 Request processed in 150us (652us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762798.080983:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 147 00000100:00100000:3.0:1789762798.080994:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074656128 00000100:00100000:3.0:1789762798.081002:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074656128 00000100:00100000:0.0:1789762798.081018:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074655744 00000100:00100000:2.0:1789762798.081021:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074656256:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.081022:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762798.081024:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074655744 00000100:00100000:3.0:1789762798.081053:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074656640 00000100:00100000:3.0:1789762798.081059:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074656640 00000100:00100000:1.0:1789762798.081090:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074656896:0@lo:400:kworker.0 00000100:00100000:2.0:1789762798.081090:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762798.081100:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:3.0:1789762798.081101:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074657024 00000100:00100000:3.0:1789762798.081113:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074657024 00000100:00100000:0.0:1789762798.081121:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074655744:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762798.081122:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074656768:0@lo:400:kworker.0 00000100:00100000:3.0:1789762798.081136:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074656384 00000100:00100000:3.0:1789762798.081140:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074656384 00002000:00000024:0.0:1789762798.081151:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3851419648 avail 3849322496 00000100:00100000:1.0:1789762798.081158:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762798.081184:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074656896 00000100:00100000:3.0:1789762798.081188:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074656896 00000100:00100000:2.0:1789762798.081198:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762798.081200:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074657408:0@lo:400:kworker.0 00000100:00100000:0.0:1789762798.081221:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074655744:12345-0@lo:13:osp-pre-1-0.0 Request processed in 106us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762798.081228:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074656768 00000100:00100000:3.0:1789762798.081236:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074656768 00000100:00100000:0.0:1789762798.081250:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000100:00100000:2.0:1789762798.081251:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074657280:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.081258:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762798.081266:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:3.0:1789762798.081279:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074656128:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762798.081290:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762798.081296:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762798.081305:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762798.081324:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762798.081329:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074656512 00000100:00100000:0.0:1789762798.081335:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074656512 00000100:00100000:3.0:1789762798.081351:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074656128:12345-0@lo:400:kworker.0 Request processed in 74us (899us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762798.081361:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 353 00000100:00100000:3.0:1789762798.081371:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00100000:1.0:1789762798.081382:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074656640:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762798.081394:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074656256 00000100:00100000:0.0:1789762798.081398:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074656256 00000100:00100000:3.0:1789762798.081411:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:2.0:1789762798.081415:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074655616:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762798.081429:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074657024:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762798.081446:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:2.0:1789762798.081451:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074656128:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.081454:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074656640:12345-0@lo:400:kworker.0 Request processed in 80us (831us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762798.081464:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074656512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.081473:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 353 00000100:00100000:1.0:1789762798.081514:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:3.0:1789762798.081522:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074657024:12345-0@lo:400:kworker.0 Request processed in 99us (794us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762798.081524:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074656384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762798.081543:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074655744:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762798.081551:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074656512:12345-0@lo:400:kworker.0 Request processed in 86us (930us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762798.081553:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00100000:0.0:1789762798.081574:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00000100:00100000:1.0:1789762798.081597:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074656384:12345-0@lo:400:kworker.0 Request processed in 71us (579us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762798.081599:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074656640:0@lo:400:kworker.0 00000100:00100000:0.0:1789762798.081618:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 236 00000100:00100000:3.0:1789762798.081619:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:0.0:1789762798.081635:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9a37450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3426:x1876702074656256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.081644:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00100000:3.0:1789762798.081647:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074656896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762798.081665:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 357 00000100:00100000:2.0:1789762798.081682:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074656768:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762798.081718:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9a37450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3426:x1876702074656256:12345-0@lo:400:kworker.0 Request processed in 85us (631us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762798.081736:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 236 00000100:00100000:3.0:1789762798.081737:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074656896:12345-0@lo:400:kworker.0 Request processed in 90us (574us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762798.081757:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074657152 00000100:00100000:3.0:1789762798.081758:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:2.0:1789762798.081762:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074656768:12345-0@lo:400:kworker.0 Request processed in 82us (569us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762798.081767:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074657152 00000100:00100000:2.0:1789762798.081773:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 357 00000100:00100000:3.0:1789762798.081834:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074656384:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.081870:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074657408 00000100:00100000:2.0:1789762798.081871:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074657280 00000100:00100000:1.0:1789762798.081877:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074657408 00000100:00100000:2.0:1789762798.081880:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074657280 00000100:00100000:3.0:1789762798.081885:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074656896:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.081951:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 136 00000100:00100000:1.0:1789762798.081963:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074657152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762798.081970:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074656512:0@lo:400:kworker.0 00000100:00100000:2.0:1789762798.081987:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 137 00000100:00100000:2.0:1789762798.082002:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074657408:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762798.082005:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9fff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074657024:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.082035:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074657152:12345-0@lo:400:kworker.0 Request processed in 73us (1303us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762798.082050:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 136 00000100:00100000:2.0:1789762798.082087:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074657408:12345-0@lo:400:kworker.0 Request processed in 85us (830us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762798.082096:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 137 00000100:00100000:1.0:1789762798.082110:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:3.0:1789762798.082119:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074657408:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.082134:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074657280:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762798.082161:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762798.082202:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074657280:12345-0@lo:400:kworker.0 Request processed in 79us (884us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762798.082218:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762798.082232:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074656256:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.082238:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00000100:00100000:2.0:1789762798.082283:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074656768:0@lo:400:kworker.0 00000100:00100000:2.0:1789762798.082324:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074657280:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.082332:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074655872 00000100:00100000:1.0:1789762798.082338:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074655872 00000100:00100000:2.0:1789762798.082358:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074657152:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.082398:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:1.0:1789762798.082414:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074655872:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789762798.082430:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3746562048 avail 3744464896 00000100:00100000:1.0:1789762798.082471:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074655872:12345-0@lo:13:osp-pre-0-0.0 Request processed in 65us (269us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762798.082478:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000100:00100000:3.0:1789762798.082520:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074655872:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762798.145040:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271392021160 time 271392103731 state 0x0 ref 1 00000100:00100000:3.0:1789762798.145065:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067344512:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762798.145100:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175359 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.145114:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762798.145118:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762798.145122:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175359 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.145140:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175359 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.148037:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175359 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.148051:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175359 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.148062:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175359 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.148071:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d108e000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762798.148076:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175359 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.148086:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175359 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.148104:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175359 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.148113:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f175359 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762798.148177:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067344512:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3121us (98429us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762798.148189:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 155 00000100:00100000:3.0:1789762798.148208:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271392021160 time 271392103731 state 0x2 ref 0 00000100:00100000:2.0:1789762798.150166:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762798.150225:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067344640 00000100:00100000:1.0:1789762798.150238:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067344640 00000100:00100000:1.0:1789762798.150308:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271392021160 time 271392103731 state 0x0 ref 1 00000100:00100000:1.0:1789762798.150328:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271492103731 time 271392103731 state 0x1 ref 1 00010000:00010000:1.0:1789762798.208068:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762798.245127:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271492103731 time 271492190875 state 0x0 ref 1 00000100:00100000:2.0:1789762798.245154:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067344640:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762798.245208:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f175360 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.245225:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762798.245229:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762798.245236:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f175360 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.245251:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f175360 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.248639:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f175360 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.248656:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f175360 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.248667:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f175360 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.248678:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762798.248685:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f175360 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.248698:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f175360 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.248721:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f175360 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.248728:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000043d89470/0xdf5a7ac59f175360 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762798.248813:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067344640:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3660us (98640us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762798.248825:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 156 00000100:00100000:1.0:1789762798.248836:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271492103731 time 271492190875 state 0x2 ref 0 00000100:00100000:0.0:1789762798.250899:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762798.250990:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067344768 00000100:00100000:3.0:1789762798.251000:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067344768 00000100:00100000:3.0:1789762798.251061:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271492103731 time 271492190875 state 0x0 ref 1 00000100:00100000:3.0:1789762798.251081:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271592190875 time 271492190875 state 0x1 ref 1 00000100:00100000:1.0:1789762798.345212:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271592190875 time 271592277690 state 0x0 ref 1 00000100:00100000:1.0:1789762798.345239:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9fff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067344768:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762798.345284:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175367 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.345298:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762798.345302:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762798.345307:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175367 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.345318:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175367 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.348268:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175367 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.348279:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175367 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.348286:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175367 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.348292:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762798.348295:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175367 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.348316:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175367 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.348330:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175367 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.348336:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175367 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762798.348385:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9fff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067344768:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3157us (97487us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762798.348392:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 157 00000100:00100000:1.0:1789762798.348400:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271592190875 time 271592277690 state 0x2 ref 0 00000100:00100000:0.0:1789762798.350305:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762798.350348:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067344896 00000100:00100000:3.0:1789762798.350356:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067344896 00000100:00100000:3.0:1789762798.350391:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271592190875 time 271592277690 state 0x0 ref 1 00000100:00100000:3.0:1789762798.350401:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271692277690 time 271592277690 state 0x1 ref 1 00000100:00100000:3.0:1789762798.445278:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271692277690 time 271692344355 state 0x0 ref 1 00000100:00100000:3.0:1789762798.445306:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067344896:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762798.445343:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f17536e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.445357:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762798.445362:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762798.445370:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f17536e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.445384:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f17536e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.448339:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f17536e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.448398:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f17536e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.448405:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f17536e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.448411:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d108e000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762798.448414:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f17536e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.448420:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f17536e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.448433:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f17536e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.448440:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba14777a/0xdf5a7ac59f17536e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762798.448489:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067344896:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3195us (98187us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762798.448495:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 158 00000100:00100000:3.0:1789762798.448508:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271692277690 time 271692344355 state 0x2 ref 0 00000100:00100000:0.0:1789762798.450463:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762798.450518:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067345024 00000100:00100000:1.0:1789762798.450526:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067345024 00000100:00100000:1.0:1789762798.450563:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271692277690 time 271692344355 state 0x0 ref 1 00000100:00100000:1.0:1789762798.450574:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271792344355 time 271692344355 state 0x1 ref 1 00000100:00100000:2.0:1789762798.545381:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271792344355 time 271792443126 state 0x0 ref 1 00000100:00100000:2.0:1789762798.545420:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067345024:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762798.545461:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003a1b75e4/0xdf5a7ac59f175375 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.545477:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762798.545482:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762798.545488:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a1b75e4/0xdf5a7ac59f175375 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.545502:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a1b75e4/0xdf5a7ac59f175375 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.548603:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003a1b75e4/0xdf5a7ac59f175375 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.548622:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a1b75e4/0xdf5a7ac59f175375 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.548630:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a1b75e4/0xdf5a7ac59f175375 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.548635:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7f27000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762798.548639:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a1b75e4/0xdf5a7ac59f175375 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.548645:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a1b75e4/0xdf5a7ac59f175375 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.548659:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a1b75e4/0xdf5a7ac59f175375 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.548665:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a1b75e4/0xdf5a7ac59f175375 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762798.548712:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067345024:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3314us (98250us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762798.548720:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 159 00000100:00100000:2.0:1789762798.548728:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271792344355 time 271792443126 state 0x2 ref 0 00000100:00100000:2.0:1789762798.550788:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762798.550838:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067345152 00000100:00100000:1.0:1789762798.550846:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067345152 00000100:00100000:1.0:1789762798.550904:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271792344355 time 271792443126 state 0x0 ref 1 00000100:00100000:1.0:1789762798.550945:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271892443126 time 271792443126 state 0x1 ref 1 00000100:00100000:1.0:1789762798.645446:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271892443126 time 271892510191 state 0x0 ref 1 00000100:00100000:1.0:1789762798.645469:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067345152:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762798.645506:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17537c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.645520:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762798.645523:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762798.645528:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17537c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.645539:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17537c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.648602:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17537c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.648617:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17537c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.648627:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17537c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.648633:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762798.648637:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17537c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.648658:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17537c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.648677:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17537c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.648687:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f17537c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762798.648739:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067345152:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3279us (97951us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762798.648748:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 160 00000100:00100000:1.0:1789762798.648755:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271892443126 time 271892510191 state 0x2 ref 0 00000100:00100000:3.0:1789762798.650534:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762798.650570:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067345280 00000100:00100000:1.0:1789762798.650579:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067345280 00000100:00100000:1.0:1789762798.650626:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271892443126 time 271892510191 state 0x0 ref 1 00000100:00100000:1.0:1789762798.650639:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271992510191 time 271892510191 state 0x1 ref 1 00000100:00100000:3.0:1789762798.745604:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271992510191 time 271992668613 state 0x0 ref 1 00000100:00100000:3.0:1789762798.745632:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067345280:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762798.745670:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f175383 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.745683:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762798.745687:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762798.745692:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f175383 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.745703:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f175383 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.748672:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f175383 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.748683:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f175383 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.748704:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f175383 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.748711:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998fa00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762798.748715:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f175383 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.748721:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f175383 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.748736:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f175383 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762798.748742:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f175383 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762798.748809:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067345280:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3188us (98274us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762798.748818:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 161 00000100:00100000:1.0:1789762798.748829:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271992510191 time 271992668613 state 0x2 ref 0 00000100:00100000:3.0:1789762798.750882:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762798.750941:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067345408 00000100:00100000:2.0:1789762798.750949:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067345408 00000100:00100000:2.0:1789762798.750985:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 271992510191 time 271992668613 state 0x0 ref 1 00000100:00100000:2.0:1789762798.750995:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272092668613 time 271992668613 state 0x1 ref 1 00000100:00100000:3.0:1789762798.845695:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272092668613 time 272092760988 state 0x0 ref 1 00000100:00100000:3.0:1789762798.845720:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067345408:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762798.845757:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17538a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.845770:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762798.845793:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762798.845798:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17538a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.845811:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17538a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.848560:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17538a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.848571:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17538a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.848579:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17538a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.848584:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762798.848588:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17538a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.848595:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17538a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.848609:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17538a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762798.848614:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17538a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762798.848665:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067345408:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2955us (97784us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762798.848672:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 162 00000100:00100000:3.0:1789762798.848680:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272092668613 time 272092760988 state 0x2 ref 0 00000100:00100000:3.0:1789762798.851013:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762798.851069:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067345536 00000100:00100000:1.0:1789762798.851078:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067345536 00000100:00100000:1.0:1789762798.851116:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272092668613 time 272092760988 state 0x0 ref 1 00000100:00100000:1.0:1789762798.851136:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272192760988 time 272092760988 state 0x1 ref 1 00000100:00100000:2.0:1789762798.945789:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272192760988 time 272192850653 state 0x0 ref 1 00000100:00100000:2.0:1789762798.945815:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067345536:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762798.945867:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f175391 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.945882:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762798.945886:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762798.945891:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f175391 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.945908:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f175391 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.948676:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f175391 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.948689:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f175391 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.948698:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f175391 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.948715:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d108ee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762798.948718:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f175391 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.948725:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f175391 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.948738:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f175391 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762798.948744:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f175391 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762798.948789:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067345536:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2985us (97776us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762798.948795:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 163 00000100:00100000:2.0:1789762798.948803:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272192760988 time 272192850653 state 0x2 ref 0 00000100:00100000:0.0:1789762798.950899:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762798.950954:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067345664 00000100:00100000:1.0:1789762798.950962:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067345664 00000100:00100000:1.0:1789762798.951003:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272192760988 time 272192850653 state 0x0 ref 1 00000100:00100000:1.0:1789762798.951014:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272292850653 time 272192850653 state 0x1 ref 1 00000100:00100000:1.0:1789762799.045891:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272292850653 time 272292955873 state 0x0 ref 1 00000100:00100000:1.0:1789762799.045953:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067345664:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:2.0:1789762799.045983:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762799.045994:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175398 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.046009:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762799.046012:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762799.046017:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175398 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.046028:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175398 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.048748:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175398 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.048771:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175398 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.048778:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175398 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.048783:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f1600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762799.048786:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175398 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.048793:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175398 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.048809:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175398 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.048817:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175398 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762799.048884:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067345664:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2977us (97986us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762799.048893:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 164 00000100:00100000:1.0:1789762799.048900:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272292850653 time 272292955873 state 0x2 ref 0 00000100:00100000:1.0:1789762799.050889:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762799.050947:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067345792 00000100:00100000:3.0:1789762799.050955:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067345792 00000100:00100000:3.0:1789762799.050992:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272292850653 time 272292955873 state 0x0 ref 1 00000100:00100000:3.0:1789762799.051002:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272392955873 time 272292955873 state 0x1 ref 1 00000100:00100000:3.0:1789762799.145952:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272392955873 time 272393016901 state 0x0 ref 1 00000100:00100000:3.0:1789762799.145982:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067345792:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762799.146025:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17539f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.146040:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762799.146044:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762799.146048:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17539f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.146059:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17539f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.152414:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17539f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.152427:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17539f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.152433:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17539f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.152438:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762799.152442:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17539f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.152448:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17539f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.152461:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17539f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.152466:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17539f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762799.152513:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067345792:12345-192.168.203.37@tcp:3:dd.500 Request processed in 6546us (101623us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762799.152521:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 165 00000100:00100000:3.0:1789762799.152535:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272392955873 time 272393016901 state 0x2 ref 0 00000100:00100000:3.0:1789762799.154437:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762799.154486:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067345920 00000100:00100000:2.0:1789762799.154494:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067345920 00000100:00100000:2.0:1789762799.154531:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272392955873 time 272393016901 state 0x0 ref 1 00000100:00100000:2.0:1789762799.154542:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272493016901 time 272393016901 state 0x1 ref 1 00010000:00010000:3.0:1789762799.232026:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762799.245991:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272493016901 time 272493059741 state 0x0 ref 1 00000100:00100000:2.0:1789762799.246012:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067345920:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762799.246046:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f1753a6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.246060:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762799.246064:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762799.246068:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f1753a6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.246078:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f1753a6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.248888:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f1753a6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.248904:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f1753a6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.248912:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f1753a6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.248943:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d108ee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762799.248947:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f1753a6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.248966:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f1753a6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.248982:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f1753a6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.248988:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000486cd48d/0xdf5a7ac59f1753a6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762799.249034:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067345920:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3029us (94597us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762799.249041:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 166 00000100:00100000:2.0:1789762799.249049:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272493016901 time 272493059741 state 0x2 ref 0 00000100:00100000:3.0:1789762799.250909:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762799.250962:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067346048 00000100:00100000:1.0:1789762799.250970:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067346048 00000100:00100000:1.0:1789762799.251010:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272493016901 time 272493059741 state 0x0 ref 1 00000100:00100000:1.0:1789762799.251021:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272593059741 time 272493059741 state 0x1 ref 1 00000100:00100000:1.0:1789762799.346061:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272593059741 time 272593125978 state 0x0 ref 1 00000100:00100000:1.0:1789762799.346086:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067346048:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762799.346124:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753ad lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.346138:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762799.346142:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762799.346146:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753ad lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.346157:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753ad lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.348900:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753ad lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.348910:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753ad lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.348947:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753ad lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.348954:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762799.348961:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753ad lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.348971:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753ad lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.348990:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753ad lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.348999:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753ad lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762799.349049:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067346048:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2974us (98141us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762799.349057:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 167 00000100:00100000:1.0:1789762799.349064:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272593059741 time 272593125978 state 0x2 ref 0 00000100:00100000:1.0:1789762799.351180:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762799.351220:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067346176 00000100:00100000:3.0:1789762799.351228:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067346176 00000100:00100000:3.0:1789762799.351262:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272593059741 time 272593125978 state 0x0 ref 1 00000100:00100000:3.0:1789762799.351272:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272693125978 time 272593125978 state 0x1 ref 1 00000100:00100000:1.0:1789762799.446142:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272693125978 time 272693206265 state 0x0 ref 1 00000100:00100000:1.0:1789762799.446166:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067346176:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762799.446214:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753b4 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.446228:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762799.446232:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762799.446236:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753b4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.446248:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753b4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.448970:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753b4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.448982:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753b4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.448989:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753b4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.448994:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762799.448999:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753b4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.449005:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753b4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.449030:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753b4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.449036:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1753b4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762799.449082:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067346176:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2925us (97903us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762799.449089:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 168 00000100:00100000:3.0:1789762799.449096:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272693125978 time 272693206265 state 0x2 ref 0 00000100:00100000:0.0:1789762799.450949:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762799.450999:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067346304 00000100:00100000:2.0:1789762799.451009:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067346304 00000100:00100000:2.0:1789762799.451054:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272693125978 time 272693206265 state 0x0 ref 1 00000100:00100000:2.0:1789762799.451069:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272793206265 time 272693206265 state 0x1 ref 1 00000100:00100000:3.0:1789762799.546221:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272793206265 time 272793288098 state 0x0 ref 1 00000100:00100000:3.0:1789762799.546248:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067346304:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762799.546284:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753bb lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.546298:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762799.546302:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762799.546306:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753bb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.546318:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753bb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.549281:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753bb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.549292:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753bb lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.549298:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753bb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.549303:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762799.549306:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753bb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.549312:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753bb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.549325:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753bb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.549330:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753bb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762799.549378:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067346304:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3141us (98459us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762799.549385:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 169 00000100:00100000:3.0:1789762799.549399:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272793206265 time 272793288098 state 0x2 ref 0 00000100:00100000:2.0:1789762799.551410:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762799.551457:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067346432 00000100:00100000:1.0:1789762799.551465:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067346432 00000100:00100000:1.0:1789762799.551503:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272793206265 time 272793288098 state 0x0 ref 1 00000100:00100000:1.0:1789762799.551515:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272893288098 time 272793288098 state 0x1 ref 1 00000100:00100000:2.0:1789762799.646326:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272893288098 time 272893384522 state 0x0 ref 1 00000100:00100000:2.0:1789762799.646352:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067346432:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762799.646402:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f88208e7/0xdf5a7ac59f1753c2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.646417:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762799.646422:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762799.646428:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f88208e7/0xdf5a7ac59f1753c2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.646441:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f88208e7/0xdf5a7ac59f1753c2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.649593:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f88208e7/0xdf5a7ac59f1753c2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.649604:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f88208e7/0xdf5a7ac59f1753c2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.649615:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f88208e7/0xdf5a7ac59f1753c2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.649623:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d108f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762799.649629:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f88208e7/0xdf5a7ac59f1753c2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.649639:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f88208e7/0xdf5a7ac59f1753c2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.649659:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f88208e7/0xdf5a7ac59f1753c2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.649667:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f88208e7/0xdf5a7ac59f1753c2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762799.649715:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067346432:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3374us (98306us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762799.649722:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 170 00000100:00100000:3.0:1789762799.649737:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272893288098 time 272893384522 state 0x2 ref 0 00000100:00100000:2.0:1789762799.651655:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762799.651690:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067346560 00000100:00100000:3.0:1789762799.651698:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067346560 00000100:00100000:3.0:1789762799.651735:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272893288098 time 272893384522 state 0x0 ref 1 00000100:00100000:3.0:1789762799.651745:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272993384522 time 272893384522 state 0x1 ref 1 00000100:00100000:1.0:1789762799.746443:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272993384522 time 272993506627 state 0x0 ref 1 00000100:00100000:1.0:1789762799.746467:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067346560:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762799.746505:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753c9 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.746519:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762799.746523:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762799.746527:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753c9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.746552:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753c9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.750451:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753c9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.750462:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753c9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.750468:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753c9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.750473:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b9be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762799.750476:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753c9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.750482:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753c9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.750494:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753c9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.750499:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753c9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762799.750559:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067346560:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4101us (98904us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762799.750568:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 171 00000100:00100000:1.0:1789762799.750576:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272993384522 time 272993506627 state 0x2 ref 0 00000100:00100000:0.0:1789762799.752473:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762799.752515:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067346688 00000100:00100000:3.0:1789762799.752523:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067346688 00000100:00100000:3.0:1789762799.752561:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 272993384522 time 272993506627 state 0x0 ref 1 00000100:00100000:3.0:1789762799.752572:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273093506627 time 272993506627 state 0x1 ref 1 00000100:00100000:3.0:1789762799.846506:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273093506627 time 273093571757 state 0x0 ref 1 00000100:00100000:3.0:1789762799.846531:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067346688:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762799.846567:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753d0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.846581:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762799.846584:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762799.846589:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753d0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.846601:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753d0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.849299:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753d0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.849310:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753d0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.849317:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753d0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.849323:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762799.849326:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753d0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.849333:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753d0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.849347:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753d0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.849353:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753d0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762799.849399:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067346688:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2880us (96927us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762799.849406:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 172 00000100:00100000:3.0:1789762799.849421:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273093506627 time 273093571757 state 0x2 ref 0 00000100:00100000:2.0:1789762799.851301:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762799.851332:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067346816 00000100:00100000:3.0:1789762799.851339:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067346816 00000100:00100000:3.0:1789762799.851372:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273093506627 time 273093571757 state 0x0 ref 1 00000100:00100000:3.0:1789762799.851382:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273193571757 time 273093571757 state 0x1 ref 1 00000100:00100000:2.0:1789762799.946605:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273193571757 time 273193670620 state 0x0 ref 1 00000100:00100000:2.0:1789762799.946630:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067346816:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762799.946668:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000028089625/0xdf5a7ac59f1753d7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.946682:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762799.946686:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762799.946690:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000028089625/0xdf5a7ac59f1753d7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.946702:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000028089625/0xdf5a7ac59f1753d7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.949597:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000028089625/0xdf5a7ac59f1753d7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.949609:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000028089625/0xdf5a7ac59f1753d7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.949616:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000028089625/0xdf5a7ac59f1753d7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.949621:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c2182400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762799.949625:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000028089625/0xdf5a7ac59f1753d7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.949631:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000028089625/0xdf5a7ac59f1753d7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.949644:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000028089625/0xdf5a7ac59f1753d7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.949650:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000028089625/0xdf5a7ac59f1753d7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762799.949713:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067346816:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3091us (98410us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762799.949722:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 173 00000100:00100000:1.0:1789762799.949733:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273193571757 time 273193670620 state 0x2 ref 0 00000100:00100000:1.0:1789762799.951705:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762799.951751:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067346944 00000100:00100000:3.0:1789762799.951760:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067346944 00000100:00100000:3.0:1789762799.951807:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273193571757 time 273193670620 state 0x0 ref 1 00000100:00100000:3.0:1789762799.951821:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273293670620 time 273193670620 state 0x1 ref 1 00000100:00100000:1.0:1789762800.046698:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273293670620 time 273293763720 state 0x0 ref 1 00000100:00100000:1.0:1789762800.046722:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067346944:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762800.046759:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753de lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.046772:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762800.046775:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762800.046780:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753de lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.046791:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753de lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.049404:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753de lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.049414:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753de lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.049428:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753de lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.049434:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b9be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762800.049438:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753de lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.049444:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753de lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.049461:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753de lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.049467:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753de lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762800.049520:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067346944:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2806us (97814us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762800.049532:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 174 00000100:00100000:1.0:1789762800.049541:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273293670620 time 273293763720 state 0x2 ref 0 00000100:00100000:1.0:1789762800.051604:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762800.051644:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067347072 00000100:00100000:3.0:1789762800.051652:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067347072 00000100:00100000:3.0:1789762800.051689:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273293670620 time 273293763720 state 0x0 ref 1 00000100:00100000:3.0:1789762800.051699:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273393763720 time 273293763720 state 0x1 ref 1 00000100:00100000:2.0:1789762800.128137:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762800.128159:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074657536:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762800.128316:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762800.128391:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074657536 00000100:00100000:0.0:1789762800.128409:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074657536 00000100:00100000:0.0:1789762800.128477:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1789762800.128488:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074657536:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762800.128563:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762800.128628:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074657536:12345-0@lo:41:osp-pre-1-0.0 Request processed in 143us (313us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762800.128637:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00000100:00100000:2.0:1789762800.128690:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074657536:0@lo:41:osp-pre-1-0.0 00000001:00080000:0.0:1789762800.141095:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967363 is committed 00000100:00100000:1.0:1789762800.146927:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273393763720 time 273393985773 state 0x0 ref 1 00000100:00100000:1.0:1789762800.146993:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067347072:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762800.147041:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753e5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.147055:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762800.147059:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762800.147064:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753e5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.147075:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753e5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.149911:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753e5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.149943:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753e5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.149950:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753e5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.149962:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b9be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762800.149966:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753e5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.149973:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753e5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.149986:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753e5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.149991:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3b1bf/0xdf5a7ac59f1753e5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.150039:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067347072:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3054us (98434us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762800.150046:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 175 00000100:00100000:3.0:1789762800.150054:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273393763720 time 273393985773 state 0x2 ref 0 00000100:00100000:2.0:1789762800.152080:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762800.152135:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067347200 00000100:00100000:1.0:1789762800.152142:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067347200 00000100:00100000:1.0:1789762800.152189:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273393763720 time 273393985773 state 0x0 ref 1 00000100:00100000:1.0:1789762800.152200:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273493985773 time 273393985773 state 0x1 ref 1 00000100:00100000:1.0:1789762800.246981:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273493985773 time 273494045863 state 0x0 ref 1 00000100:00100000:1.0:1789762800.247006:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067347200:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762800.247042:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000892db4de/0xdf5a7ac59f1753ec lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.247063:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762800.247067:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762800.247074:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000892db4de/0xdf5a7ac59f1753ec lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.247104:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000892db4de/0xdf5a7ac59f1753ec lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.249889:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000892db4de/0xdf5a7ac59f1753ec lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.249904:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000892db4de/0xdf5a7ac59f1753ec lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.249913:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000892db4de/0xdf5a7ac59f1753ec lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.249942:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b9b800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762800.249948:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000892db4de/0xdf5a7ac59f1753ec lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.249959:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000892db4de/0xdf5a7ac59f1753ec lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.249979:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000892db4de/0xdf5a7ac59f1753ec lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.249987:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000892db4de/0xdf5a7ac59f1753ec lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762800.250043:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067347200:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3045us (97962us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762800.250053:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 176 00000100:00100000:1.0:1789762800.250064:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273493985773 time 273494045863 state 0x2 ref 0 00000100:00100000:2.0:1789762800.252000:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762800.252035:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067347328 00000100:00100000:1.0:1789762800.252042:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067347328 00000100:00100000:1.0:1789762800.252077:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273493985773 time 273494045863 state 0x0 ref 1 00000100:00100000:1.0:1789762800.252087:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273594045863 time 273494045863 state 0x1 ref 1 00010000:00010000:3.0:1789762800.256025:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762800.347079:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273594045863 time 273594144615 state 0x0 ref 1 00000100:00100000:2.0:1789762800.347105:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067347328:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762800.347142:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f1753f3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.347156:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762800.347159:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762800.347164:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f1753f3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.347175:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f1753f3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.350100:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f1753f3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.350114:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f1753f3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.350137:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f1753f3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.350143:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762800.350146:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f1753f3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.350153:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f1753f3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.350167:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f1753f3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.350173:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f1753f3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.350221:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067347328:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3127us (98221us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762800.350229:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 177 00000100:00100000:3.0:1789762800.350243:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273594045863 time 273594144615 state 0x2 ref 0 00000100:00100000:3.0:1789762800.352015:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762800.352063:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067347456 00000100:00100000:1.0:1789762800.352072:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067347456 00000100:00100000:1.0:1789762800.352112:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273594045863 time 273594144615 state 0x0 ref 1 00000100:00100000:1.0:1789762800.352122:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273694144615 time 273594144615 state 0x1 ref 1 00000100:00100000:3.0:1789762800.447172:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273694144615 time 273694236155 state 0x0 ref 1 00000100:00100000:3.0:1789762800.447197:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067347456:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762800.447246:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753fa lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.447260:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.447264:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.447268:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753fa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.447279:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753fa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.449941:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753fa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.449992:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753fa lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.450003:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753fa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.450012:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762800.450016:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753fa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.450024:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753fa lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.450049:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753fa lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.450055:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f1753fa lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.450113:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067347456:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2926us (98097us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762800.450120:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 178 00000100:00100000:3.0:1789762800.450128:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273694144615 time 273694236155 state 0x2 ref 0 00000100:00100000:3.0:1789762800.452174:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762800.452222:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067347584 00000100:00100000:1.0:1789762800.452229:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067347584 00000100:00100000:1.0:1789762800.452266:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273694144615 time 273694236155 state 0x0 ref 1 00000100:00100000:1.0:1789762800.452276:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273794236155 time 273694236155 state 0x1 ref 1 00000100:00100000:3.0:1789762800.547270:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273794236155 time 273794335673 state 0x0 ref 1 00000100:00100000:3.0:1789762800.547295:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067347584:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762800.547333:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175401 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.547347:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.547350:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.547355:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175401 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.547366:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175401 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.550234:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175401 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.550246:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175401 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.550256:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175401 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.550266:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762800.550271:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175401 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.550282:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175401 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.550302:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175401 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.550312:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d3a0567/0xdf5a7ac59f175401 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.550377:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067347584:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3092us (98203us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762800.550384:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 179 00000100:00100000:3.0:1789762800.550399:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273794236155 time 273794335673 state 0x2 ref 0 00000100:00100000:3.0:1789762800.552511:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762800.552556:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067347712 00000100:00100000:2.0:1789762800.552564:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067347712 00000100:00100000:2.0:1789762800.552610:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273794236155 time 273794335673 state 0x0 ref 1 00000100:00100000:2.0:1789762800.552636:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273894335673 time 273794335673 state 0x1 ref 1 00000100:00100000:2.0:1789762800.647334:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273894335673 time 273894395503 state 0x0 ref 1 00000100:00100000:2.0:1789762800.647365:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067347712:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762800.647404:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175408 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.647417:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762800.647421:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762800.647425:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175408 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.647436:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175408 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.650294:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175408 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.650305:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175408 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.650311:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175408 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.650341:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762800.650346:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175408 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.650357:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175408 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.650375:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175408 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.650382:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175408 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762800.650430:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067347712:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3076us (97919us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762800.650436:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 180 00000100:00100000:2.0:1789762800.650444:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273894335673 time 273894395503 state 0x2 ref 0 00000100:00100000:3.0:1789762800.652290:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762800.652347:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067347840 00000100:00100000:1.0:1789762800.652372:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067347840 00000100:00100000:1.0:1789762800.652417:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273894335673 time 273894395503 state 0x0 ref 1 00000100:00100000:1.0:1789762800.652432:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273994395503 time 273894395503 state 0x1 ref 1 00000100:00100000:2.0:1789762800.747431:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273994395503 time 273994492000 state 0x0 ref 1 00000100:00100000:2.0:1789762800.747468:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067347840:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762800.747510:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17540f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.747547:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762800.747552:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762800.747560:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17540f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.747574:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17540f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.750523:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17540f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.750538:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17540f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.750547:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17540f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.750555:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762800.750560:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17540f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.750571:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17540f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.750590:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17540f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.750609:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17540f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762800.750665:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067347840:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3207us (98373us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762800.750674:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 181 00000100:00100000:2.0:1789762800.750684:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273994395503 time 273994492000 state 0x2 ref 0 00000100:00100000:1.0:1789762800.752565:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762800.752614:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067347968 00000100:00100000:3.0:1789762800.752621:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067347968 00000100:00100000:3.0:1789762800.752657:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 273994395503 time 273994492000 state 0x0 ref 1 00000100:00100000:3.0:1789762800.752667:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274094492000 time 273994492000 state 0x1 ref 1 00000100:00100000:1.0:1789762800.847530:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274094492000 time 274094596890 state 0x0 ref 1 00000100:00100000:1.0:1789762800.847553:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067347968:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762800.847589:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175416 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.847603:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762800.847606:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762800.847611:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175416 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.847623:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175416 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.850017:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175416 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.850029:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175416 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.850057:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175416 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.850067:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762800.850072:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175416 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.850083:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175416 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.850102:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175416 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.850113:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079f46bb8/0xdf5a7ac59f175416 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762800.850165:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067347968:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2620us (97600us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762800.850172:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 182 00000100:00100000:1.0:1789762800.850180:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274094492000 time 274094596890 state 0x2 ref 0 00000100:00100000:1.0:1789762800.852096:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762800.852153:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067348096 00000100:00100000:2.0:1789762800.852163:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067348096 00000100:00100000:2.0:1789762800.852209:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274094492000 time 274094596890 state 0x0 ref 1 00000100:00100000:2.0:1789762800.852225:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274194596890 time 274094596890 state 0x1 ref 1 00000100:00100000:3.0:1789762800.947617:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274194596890 time 274194674665 state 0x0 ref 1 00000100:00100000:3.0:1789762800.947643:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067348096:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762800.947680:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17541d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.947694:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.947698:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.947703:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17541d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.947714:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17541d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.950748:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17541d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.950778:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17541d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.950790:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17541d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.950799:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762800.950804:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17541d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.950814:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17541d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.950832:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17541d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.950857:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17541d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.950958:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067348096:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3283us (98820us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762800.950970:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 183 00000100:00100000:3.0:1789762800.950982:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274194596890 time 274194674665 state 0x2 ref 0 00000100:00100000:3.0:1789762800.953001:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762800.953048:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067348224 00000100:00100000:2.0:1789762800.953056:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067348224 00000100:00100000:2.0:1789762800.953092:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274194596890 time 274194674665 state 0x0 ref 1 00000100:00100000:2.0:1789762800.953103:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274294674665 time 274194674665 state 0x1 ref 1 00000100:00100000:2.0:1789762801.047675:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274294674665 time 274294741322 state 0x0 ref 1 00000100:00100000:2.0:1789762801.047716:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067348224:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:0.0:1789762801.047761:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789762801.047768:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175424 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.047782:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762801.047786:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762801.047790:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175424 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.047801:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175424 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.050439:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175424 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.050453:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175424 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.050464:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175424 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.050473:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762801.050478:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175424 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.050488:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175424 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.050523:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175424 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.050532:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175424 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762801.050588:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067348224:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2897us (97587us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762801.050600:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 184 00000100:00100000:2.0:1789762801.050610:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274294674665 time 274294741322 state 0x2 ref 0 00000100:00100000:3.0:1789762801.052429:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762801.052470:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067348352 00000100:00100000:1.0:1789762801.052478:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067348352 00000100:00100000:1.0:1789762801.052513:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274294674665 time 274294741322 state 0x0 ref 1 00000100:00100000:1.0:1789762801.052530:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274394741322 time 274294741322 state 0x1 ref 1 00000100:00100000:2.0:1789762801.147752:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274394741322 time 274394820470 state 0x0 ref 1 00000100:00100000:2.0:1789762801.147773:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067348352:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762801.147809:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17542b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.147823:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762801.147827:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762801.147832:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17542b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.147860:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17542b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.151437:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17542b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.151452:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17542b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.151460:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17542b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.151469:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762801.151477:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17542b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.151489:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17542b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.151516:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17542b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.151528:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f17542b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762801.151610:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067348352:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3839us (99177us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762801.151626:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 185 00000100:00100000:2.0:1789762801.151643:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274394741322 time 274394820470 state 0x2 ref 0 00000100:00100000:2.0:1789762801.152163:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff70e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074657664:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762801.152227:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762801.152321:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074657664 00000100:00100000:0.0:1789762801.152335:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074657664 00000100:00100000:0.0:1789762801.152414:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:0.0:1789762801.152430:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3426:x1876702074657664:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762801.152475:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762801.152564:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3426:x1876702074657664:12345-0@lo:41:osp-pre-0-1.0 Request processed in 139us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762801.152573:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00100000:2.0:1789762801.152816:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff70e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074657664:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762801.154209:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762801.154465:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067348480 00000100:00100000:2.0:1789762801.154473:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067348480 00000100:00100000:2.0:1789762801.154536:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274394741322 time 274394820470 state 0x0 ref 1 00000100:00100000:2.0:1789762801.154552:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274494820470 time 274394820470 state 0x1 ref 1 00000001:00080000:2.0:1789762801.180609:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967377 is committed 00000001:00080000:0.0:1789762801.182221:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967673 is committed 00000100:00100000:3.0:1789762801.247858:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274494820470 time 274494909674 state 0x0 ref 1 00000100:00100000:3.0:1789762801.247879:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067348480:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762801.247914:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175432 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.247946:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.247949:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.247955:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175432 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.247967:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175432 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250600:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175432 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250613:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175432 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250623:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175432 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250632:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762801.250638:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175432 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250657:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175432 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250676:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175432 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250686:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175432 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.250747:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067348480:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2873us (96541us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762801.250756:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 186 00000100:00100000:3.0:1789762801.250767:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274494820470 time 274494909674 state 0x2 ref 0 00000100:00100000:3.0:1789762801.252590:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762801.252627:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067348608 00000100:00100000:2.0:1789762801.252635:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067348608 00000100:00100000:2.0:1789762801.252670:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274494820470 time 274494909674 state 0x0 ref 1 00000100:00100000:2.0:1789762801.252681:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274594909674 time 274494909674 state 0x1 ref 1 00010000:00010000:1.0:1789762801.280035:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762801.347963:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274594909674 time 274595029330 state 0x0 ref 1 00000100:00100000:3.0:1789762801.347989:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067348608:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762801.348025:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175439 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.348039:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.348043:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.348047:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175439 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.348072:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175439 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.352077:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175439 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.352090:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175439 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.352097:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175439 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.352102:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762801.352105:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175439 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.352111:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175439 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.352124:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175439 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.352129:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f175439 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762801.352178:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067348608:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4199us (99588us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762801.352193:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 187 00000100:00100000:1.0:1789762801.352202:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274594909674 time 274595029330 state 0x2 ref 0 00000100:00100000:3.0:1789762801.354044:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762801.354096:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067348736 00000100:00100000:2.0:1789762801.354104:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067348736 00000100:00100000:2.0:1789762801.354145:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274594909674 time 274595029330 state 0x0 ref 1 00000100:00100000:2.0:1789762801.354161:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274695029330 time 274595029330 state 0x1 ref 1 00000100:00100000:2.0:1789762801.448022:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274695029330 time 274695086479 state 0x0 ref 1 00000100:00100000:2.0:1789762801.448058:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067348736:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762801.448103:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175440 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.448117:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762801.448121:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762801.448126:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175440 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.448137:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175440 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.451068:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175440 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.451090:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175440 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.451103:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175440 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.451112:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762801.451119:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175440 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.451148:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175440 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.451171:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175440 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.451181:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175440 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762801.451287:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067348736:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3231us (97238us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762801.451301:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 188 00000100:00100000:0.0:1789762801.451315:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274695029330 time 274695086479 state 0x2 ref 0 00000100:00100000:3.0:1789762801.453160:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762801.453210:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067348864 00000100:00100000:1.0:1789762801.453219:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067348864 00000100:00100000:1.0:1789762801.453261:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274695029330 time 274695086479 state 0x0 ref 1 00000100:00100000:1.0:1789762801.453272:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274795086479 time 274695086479 state 0x1 ref 1 00000100:00100000:1.0:1789762801.548080:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274795086479 time 274795146219 state 0x0 ref 1 00000100:00100000:1.0:1789762801.548105:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067348864:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762801.548142:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175447 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.548155:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762801.548159:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762801.548164:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175447 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.548175:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175447 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.551012:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175447 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.551033:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175447 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.551040:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175447 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.551045:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b11200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762801.551049:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175447 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.551055:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175447 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.551068:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175447 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.551072:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe0bc0df/0xdf5a7ac59f175447 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762801.551126:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067348864:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3032us (97969us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762801.551133:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 189 00000100:00100000:1.0:1789762801.551141:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274795086479 time 274795146219 state 0x2 ref 0 00000100:00100000:3.0:1789762801.552981:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762801.553012:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067348992 00000100:00100000:1.0:1789762801.553026:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067348992 00000100:00100000:1.0:1789762801.553060:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274795086479 time 274795146219 state 0x0 ref 1 00000100:00100000:1.0:1789762801.553070:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274895146219 time 274795146219 state 0x1 ref 1 00000100:00100000:3.0:1789762801.648166:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274895146219 time 274895232422 state 0x0 ref 1 00000100:00100000:3.0:1789762801.648192:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067348992:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762801.648229:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f17544e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.648243:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.648247:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.648251:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f17544e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.648262:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f17544e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.650970:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f17544e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.650981:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f17544e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.650988:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f17544e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.650993:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762801.650997:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f17544e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.651003:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f17544e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.651031:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f17544e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.651037:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f17544e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762801.651093:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067348992:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2911us (98112us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762801.651099:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 190 00000100:00100000:2.0:1789762801.651106:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274895146219 time 274895232422 state 0x2 ref 0 00000100:00100000:2.0:1789762801.652988:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762801.653050:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067349120 00000100:00100000:0.0:1789762801.653058:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067349120 00000100:00100000:0.0:1789762801.653093:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274895146219 time 274895232422 state 0x0 ref 1 00000100:00100000:0.0:1789762801.653104:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274995232422 time 274895232422 state 0x1 ref 1 00000100:00100000:2.0:1789762801.748269:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274995232422 time 274995331269 state 0x0 ref 1 00000100:00100000:2.0:1789762801.748294:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067349120:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762801.748335:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f175455 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.748354:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762801.748360:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762801.748368:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f175455 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.748410:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f175455 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.753191:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f175455 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.753212:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f175455 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.753224:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f175455 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.753232:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762801.753237:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f175455 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.753245:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f175455 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.753261:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f175455 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.753268:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f175455 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762801.753324:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067349120:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5037us (100335us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762801.753333:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 191 00000100:00100000:2.0:1789762801.753351:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274995232422 time 274995331269 state 0x2 ref 0 00000100:00100000:2.0:1789762801.755701:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762801.755766:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067349248 00000100:00100000:1.0:1789762801.755777:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067349248 00000100:00100000:1.0:1789762801.755829:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 274995232422 time 274995331269 state 0x0 ref 1 00000100:00100000:1.0:1789762801.755861:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275095331269 time 274995331269 state 0x1 ref 1 00000100:00100000:2.0:1789762801.848368:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275095331269 time 275095431442 state 0x0 ref 1 00000100:00100000:2.0:1789762801.848417:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067349248:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762801.848465:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17545c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.848483:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762801.848488:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762801.848494:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17545c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.848511:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17545c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.851781:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17545c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.851797:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17545c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.851822:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17545c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.851848:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762801.851854:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17545c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.851879:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17545c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.851899:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17545c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.851906:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021769b91/0xdf5a7ac59f17545c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762801.852009:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067349248:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3622us (96307us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762801.852019:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 192 00000100:00100000:0.0:1789762801.852028:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275095331269 time 275095431442 state 0x2 ref 0 00000100:00100000:3.0:1789762801.855001:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762801.855050:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067349376 00000100:00100000:2.0:1789762801.855060:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067349376 00000100:00100000:2.0:1789762801.855100:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275095331269 time 275095431442 state 0x0 ref 1 00000100:00100000:2.0:1789762801.855112:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275195431442 time 275095431442 state 0x1 ref 1 00000100:00100000:1.0:1789762801.948464:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275195431442 time 275195526345 state 0x0 ref 1 00000100:00100000:1.0:1789762801.948492:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067349376:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762801.948534:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f175463 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.948550:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762801.948554:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762801.948559:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f175463 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.948572:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f175463 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.952279:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f175463 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.952310:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f175463 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.952335:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f175463 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.952343:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e1b10400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762801.952348:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f175463 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.952356:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f175463 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.952373:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f175463 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.952379:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff837f62/0xdf5a7ac59f175463 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.952459:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067349376:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3963us (97444us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762801.952479:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 193 00000100:00100000:3.0:1789762801.952489:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275195431442 time 275195526345 state 0x2 ref 0 00000100:00100000:3.0:1789762801.955121:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762801.955182:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067349504 00000100:00100000:2.0:1789762801.955192:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067349504 00000100:00100000:2.0:1789762801.955237:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275195431442 time 275195526345 state 0x0 ref 1 00000100:00100000:2.0:1789762801.955263:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275295526345 time 275195526345 state 0x1 ref 1 00000100:00100000:3.0:1789762802.048572:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275295526345 time 275295628952 state 0x0 ref 1 00000100:00100000:3.0:1789762802.048604:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067349504:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762802.048659:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000052ff50c9/0xdf5a7ac59f17546a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.048672:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762802.048676:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762802.048681:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000052ff50c9/0xdf5a7ac59f17546a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.048693:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052ff50c9/0xdf5a7ac59f17546a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.051529:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000052ff50c9/0xdf5a7ac59f17546a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.051538:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000052ff50c9/0xdf5a7ac59f17546a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.051544:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000052ff50c9/0xdf5a7ac59f17546a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.051549:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c7625800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762802.051552:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000052ff50c9/0xdf5a7ac59f17546a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.051558:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000052ff50c9/0xdf5a7ac59f17546a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.051587:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000052ff50c9/0xdf5a7ac59f17546a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.051597:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052ff50c9/0xdf5a7ac59f17546a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762802.051664:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067349504:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3074us (96544us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762802.051672:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 194 00000100:00100000:3.0:1789762802.051680:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275295526345 time 275295628952 state 0x2 ref 0 00000100:00100000:3.0:1789762802.053954:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762802.054014:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067349632 00000100:00100000:0.0:1789762802.054030:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067349632 00000100:00100000:0.0:1789762802.054102:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275295526345 time 275295628952 state 0x0 ref 1 00000100:00100000:0.0:1789762802.054134:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275395628952 time 275295628952 state 0x1 ref 1 00000100:00100000:2.0:1789762802.120332:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762802.120364:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762802.120381:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067349760 00000100:00100000:2.0:1789762802.120388:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762802.120392:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067349760 00000100:00100000:1.0:1789762802.120413:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067349888 00000100:00100000:2.0:1789762802.120422:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762802.120431:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067349888 00000100:00100000:3.0:1789762802.120443:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 333 00000100:00100000:2.0:1789762802.120451:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762802.120456:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067349760:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762802.120469:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067350144 00000100:00100000:0.0:1789762802.120478:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067350144 00000100:00100000:2.0:1789762802.120486:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762802.120488:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067350016 00000100:00100000:1.0:1789762802.120493:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067350016 00000100:00100000:2.0:1789762802.120520:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067350400 00000100:00100000:2.0:1789762802.120526:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067350400 00000100:00100000:1.0:1789762802.120529:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067350272 00000100:00100000:0.0:1789762802.120533:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 139 00000100:00100000:1.0:1789762802.120535:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067350272 00000100:00100000:3.0:1789762802.120536:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067349760:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 82us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762802.120550:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 333 00000100:00100000:0.0:1789762802.120551:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067350144:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762802.120560:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 358 00000100:00100000:2.0:1789762802.120572:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2395:x1876702067349888:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762802.120584:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 359 00000100:00100000:0.0:1789762802.120592:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067350144:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 49us (201us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762802.120600:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067350016:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762802.120601:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 139 00000100:00100000:2.0:1789762802.120610:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2395:x1876702067349888:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 41us (243us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762802.120617:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 358 00000100:00080000:0.0:1789762802.120628:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762802.120629:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 360 00000100:00080000:0.0:1789762802.120634:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762802.120640:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762802.120641:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 361 00000100:00100000:1.0:1789762802.120648:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067350016:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 48us (218us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762802.120648:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067350400:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762802.120658:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067350272:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762802.120665:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 359 00000100:00100000:3.0:1789762802.120695:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067350400:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 55us (242us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762802.120704:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067350272:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 51us (219us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762802.120718:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 360 00000100:00100000:2.0:1789762802.120725:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 361 00000100:00100000:0.0:1789762802.148623:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275395628952 time 275395689027 state 0x0 ref 1 00000100:00100000:0.0:1789762802.148643:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067349632:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762802.148698:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ca951997/0xdf5a7ac59f175471 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.148721:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762802.148727:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762802.148736:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca951997/0xdf5a7ac59f175471 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.148756:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca951997/0xdf5a7ac59f175471 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.151856:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ca951997/0xdf5a7ac59f175471 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.151870:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca951997/0xdf5a7ac59f175471 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.151881:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca951997/0xdf5a7ac59f175471 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.151887:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d925da00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762802.151890:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca951997/0xdf5a7ac59f175471 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.151896:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca951997/0xdf5a7ac59f175471 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.151909:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca951997/0xdf5a7ac59f175471 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.151915:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca951997/0xdf5a7ac59f175471 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762802.151976:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067349632:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3338us (98022us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762802.151983:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 195 00000100:00100000:0.0:1789762802.151996:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275395628952 time 275395689027 state 0x2 ref 0 00000100:00100000:0.0:1789762802.153981:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762802.154040:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067350528 00000100:00100000:3.0:1789762802.154048:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067350528 00000100:00100000:3.0:1789762802.154091:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275395628952 time 275395689027 state 0x0 ref 1 00000100:00100000:3.0:1789762802.154105:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275495689027 time 275395689027 state 0x1 ref 1 00000100:00100000:1.0:1789762802.248757:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275495689027 time 275495816729 state 0x0 ref 1 00000100:00100000:1.0:1789762802.248789:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067350528:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762802.248851:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175478 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.248871:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.248877:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.248885:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175478 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.248901:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175478 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.252146:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175478 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.252177:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175478 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.252184:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175478 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.252191:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762802.252195:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175478 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.252202:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175478 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.252217:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175478 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.252223:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175478 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.252277:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067350528:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3500us (98298us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762802.252285:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 196 00000100:00100000:1.0:1789762802.252293:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275495689027 time 275495816729 state 0x2 ref 0 00000100:00100000:3.0:1789762802.254823:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762802.254895:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067350656 00000100:00100000:1.0:1789762802.254910:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067350656 00000100:00100000:1.0:1789762802.254965:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275495689027 time 275495816729 state 0x0 ref 1 00000100:00100000:1.0:1789762802.254977:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275595816729 time 275495816729 state 0x1 ref 1 00010000:00010000:1.0:1789762802.304040:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762802.348850:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275595816729 time 275595904000 state 0x0 ref 1 00000100:00100000:2.0:1789762802.348878:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067350656:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762802.348949:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f17547f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.348963:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.348967:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.348971:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f17547f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.348982:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f17547f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.351969:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f17547f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.351985:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f17547f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.351996:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f17547f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.352005:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa6600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762802.352011:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f17547f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.352020:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f17547f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.352036:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f17547f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.352042:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f5b07d91/0xdf5a7ac59f17547f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762802.352093:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067350656:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3226us (97271us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762802.352109:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 197 00000100:00100000:0.0:1789762802.352118:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275595816729 time 275595904000 state 0x2 ref 0 00000100:00100000:2.0:1789762802.354137:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762802.354181:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067350784 00000100:00100000:0.0:1789762802.354199:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067350784 00000100:00100000:0.0:1789762802.354244:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275595816729 time 275595904000 state 0x0 ref 1 00000100:00100000:0.0:1789762802.354259:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275695904000 time 275595904000 state 0x1 ref 1 00000100:00100000:2.0:1789762802.448954:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275695904000 time 275696019289 state 0x0 ref 1 00000100:00100000:2.0:1789762802.448980:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067350784:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762802.449016:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175486 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.449029:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.449033:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.449037:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175486 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.449048:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175486 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.451946:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175486 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.451960:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175486 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.451971:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175486 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.451978:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762802.451982:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175486 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.451989:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175486 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.452004:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175486 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.452009:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175486 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762802.452055:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067350784:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3085us (97919us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762802.452062:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 198 00000100:00100000:2.0:1789762802.452077:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275695904000 time 275696019289 state 0x2 ref 0 00000100:00100000:2.0:1789762802.454014:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762802.454065:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067350912 00000100:00100000:1.0:1789762802.454073:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067350912 00000100:00100000:1.0:1789762802.454115:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275695904000 time 275696019289 state 0x0 ref 1 00000100:00100000:1.0:1789762802.454125:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275796019289 time 275696019289 state 0x1 ref 1 00000100:00100000:0.0:1789762802.549036:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275796019289 time 275796100460 state 0x0 ref 1 00000100:00100000:0.0:1789762802.549060:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067350912:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762802.549098:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008684e667/0xdf5a7ac59f17548d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.549112:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762802.549124:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762802.549129:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008684e667/0xdf5a7ac59f17548d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.549140:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008684e667/0xdf5a7ac59f17548d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.553247:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008684e667/0xdf5a7ac59f17548d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.553279:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008684e667/0xdf5a7ac59f17548d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.553286:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008684e667/0xdf5a7ac59f17548d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.553292:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d10aec00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762802.553295:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008684e667/0xdf5a7ac59f17548d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.553301:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008684e667/0xdf5a7ac59f17548d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.553315:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008684e667/0xdf5a7ac59f17548d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.553320:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008684e667/0xdf5a7ac59f17548d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762802.553369:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067350912:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4318us (99356us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762802.553375:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 199 00000100:00100000:0.0:1789762802.553384:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275796019289 time 275796100460 state 0x2 ref 0 00000100:00100000:3.0:1789762802.555572:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762802.555637:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067351040 00000100:00100000:0.0:1789762802.555652:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067351040 00000100:00100000:0.0:1789762802.555686:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275796019289 time 275796100460 state 0x0 ref 1 00000100:00100000:0.0:1789762802.555697:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275896100460 time 275796100460 state 0x1 ref 1 00000100:00100000:3.0:1789762802.649104:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275896100460 time 275896170570 state 0x0 ref 1 00000100:00100000:3.0:1789762802.649127:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067351040:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762802.649162:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008450e79e/0xdf5a7ac59f175494 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.649176:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762802.649179:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762802.649184:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008450e79e/0xdf5a7ac59f175494 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.649195:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008450e79e/0xdf5a7ac59f175494 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.651867:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008450e79e/0xdf5a7ac59f175494 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.651878:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008450e79e/0xdf5a7ac59f175494 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.651886:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008450e79e/0xdf5a7ac59f175494 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.651892:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d925cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762802.651896:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008450e79e/0xdf5a7ac59f175494 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.651902:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008450e79e/0xdf5a7ac59f175494 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.651959:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008450e79e/0xdf5a7ac59f175494 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.651965:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008450e79e/0xdf5a7ac59f175494 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.652018:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067351040:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2899us (96446us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762802.652025:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 200 00000100:00100000:1.0:1789762802.652036:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275896100460 time 275896170570 state 0x2 ref 0 00000100:00100000:3.0:1789762802.654006:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762802.654046:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067351168 00000100:00100000:2.0:1789762802.654054:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067351168 00000100:00100000:2.0:1789762802.654089:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275896100460 time 275896170570 state 0x0 ref 1 00000100:00100000:2.0:1789762802.654099:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275996170570 time 275896170570 state 0x1 ref 1 00000100:00100000:3.0:1789762802.749187:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275996170570 time 275996252516 state 0x0 ref 1 00000100:00100000:3.0:1789762802.749234:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067351168:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762802.749276:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000017b758b6/0xdf5a7ac59f17549b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.749291:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762802.749294:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762802.749298:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017b758b6/0xdf5a7ac59f17549b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.749309:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017b758b6/0xdf5a7ac59f17549b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.751832:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000017b758b6/0xdf5a7ac59f17549b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.751842:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000017b758b6/0xdf5a7ac59f17549b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.751849:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000017b758b6/0xdf5a7ac59f17549b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.751854:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d925dc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762802.751858:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000017b758b6/0xdf5a7ac59f17549b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.751864:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000017b758b6/0xdf5a7ac59f17549b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.751886:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000017b758b6/0xdf5a7ac59f17549b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762802.751892:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017b758b6/0xdf5a7ac59f17549b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762802.751964:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067351168:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2740us (97957us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762802.751971:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 201 00000100:00100000:3.0:1789762802.751979:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275996170570 time 275996252516 state 0x2 ref 0 00000100:00100000:3.0:1789762802.753775:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762802.753867:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067351296 00000100:00100000:0.0:1789762802.753876:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067351296 00000100:00100000:0.0:1789762802.753912:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 275996170570 time 275996252516 state 0x0 ref 1 00000100:00100000:0.0:1789762802.753948:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276096252516 time 275996252516 state 0x1 ref 1 00000100:00100000:2.0:1789762802.849287:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276096252516 time 276096350987 state 0x0 ref 1 00000100:00100000:2.0:1789762802.849319:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067351296:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762802.849367:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1754a2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.849386:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762802.849390:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.849395:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1754a2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.849409:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1754a2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.852389:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1754a2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.852401:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1754a2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.852408:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1754a2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.852413:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762802.852417:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1754a2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.852423:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1754a2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.852454:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1754a2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762802.852460:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1754a2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762802.852510:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067351296:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3204us (98733us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762802.852518:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 202 00000100:00100000:0.0:1789762802.852525:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276096252516 time 276096350987 state 0x2 ref 0 00000100:00100000:3.0:1789762802.854478:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762802.854521:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067351424 00000100:00100000:1.0:1789762802.854529:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067351424 00000100:00100000:1.0:1789762802.854567:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276096252516 time 276096350987 state 0x0 ref 1 00000100:00100000:1.0:1789762802.854577:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276196350987 time 276096350987 state 0x1 ref 1 00000100:00100000:1.0:1789762802.949344:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276196350987 time 276196409004 state 0x0 ref 1 00000100:00100000:1.0:1789762802.949371:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067351424:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762802.949408:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754a9 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.949423:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.949440:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.949446:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754a9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.949457:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754a9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.952747:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754a9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.952760:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754a9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.952768:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754a9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.952773:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762802.952777:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754a9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.952783:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754a9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.952796:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754a9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.952801:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754a9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.952873:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067351424:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3513us (98395us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762802.952883:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 203 00000100:00100000:1.0:1789762802.952892:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276196350987 time 276196409004 state 0x2 ref 0 00000100:00100000:3.0:1789762802.954712:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762802.954755:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067351552 00000100:00100000:1.0:1789762802.954763:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067351552 00000100:00100000:1.0:1789762802.954805:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276196350987 time 276196409004 state 0x0 ref 1 00000100:00100000:1.0:1789762802.954834:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276296409004 time 276196409004 state 0x1 ref 1 00000100:00100000:2.0:1789762803.049484:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276296409004 time 276296548811 state 0x0 ref 1 00000100:00100000:2.0:1789762803.049524:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067351552:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762803.049578:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789762803.049588:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789762803.049597:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762803.049605:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762803.049613:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.049628:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.052543:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.052554:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.052561:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.052567:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5861a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762803.052570:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.052591:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.052610:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.052620:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762803.052674:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067351552:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3174us (97962us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762803.052681:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 204 00000100:00100000:0.0:1789762803.052689:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276296409004 time 276296548811 state 0x2 ref 0 00000100:00100000:3.0:1789762803.054468:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762803.054504:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067351680 00000100:00100000:0.0:1789762803.054512:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067351680 00000100:00100000:0.0:1789762803.054549:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276296409004 time 276296548811 state 0x0 ref 1 00000100:00100000:0.0:1789762803.054560:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276396548811 time 276296548811 state 0x1 ref 1 00000100:00100000:2.0:1789762803.149587:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276396548811 time 276396647369 state 0x0 ref 1 00000100:00100000:2.0:1789762803.149619:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067351680:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762803.149664:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.149679:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762803.149683:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762803.149688:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.149700:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.154023:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.154035:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.154043:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.154049:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5861a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762803.154063:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.154071:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.154086:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.154092:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1754b7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762803.154148:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067351680:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4541us (99679us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762803.154157:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 205 00000100:00100000:3.0:1789762803.154165:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276396548811 time 276396647369 state 0x2 ref 0 00000100:00100000:3.0:1789762803.156039:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762803.156089:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067351808 00000100:00100000:2.0:1789762803.156105:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067351808 00000100:00100000:2.0:1789762803.156146:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276396548811 time 276396647369 state 0x0 ref 1 00000100:00100000:2.0:1789762803.156160:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276496647369 time 276396647369 state 0x1 ref 1 00000100:00100000:0.0:1789762803.200060:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074657792:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762803.200100:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074657920:0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.200142:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762803.200158:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762803.200187:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074657792 00000100:00100000:1.0:1789762803.200194:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074657792 00000100:00100000:0.0:1789762803.200205:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074658304:0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.200223:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074658432:0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.200249:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762803.200268:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762803.200280:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff70ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074658176:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762803.200287:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:1.0:1789762803.200299:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074657792:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762803.200299:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074658944:0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.200302:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074658816:0@lo:400:kworker.0 00000100:00100000:3.0:1789762803.200319:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:1.0:1789762803.200348:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3754950656 avail 3752853504 00000100:00100000:3.0:1789762803.200349:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff70d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074658560:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762803.200359:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762803.200360:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762803.200385:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762803.200403:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8ab100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074659328:0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.200413:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074659456:0@lo:400:kworker.0 00000100:00100000:3.0:1789762803.200413:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074659200:0@lo:400:kworker.0 00000100:00100000:1.0:1789762803.200424:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074657792:12345-0@lo:13:osp-pre-0-0.0 Request processed in 125us (285us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762803.200436:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:0.0:1789762803.200437:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762803.200458:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d252ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074658048:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762803.200461:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074658176 00000100:00100000:1.0:1789762803.200467:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074658176 00000100:00100000:2.0:1789762803.200472:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762803.200474:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074659840:0@lo:400:kworker.0 00000100:00100000:3.0:1789762803.200491:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762803.200497:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074658560 00000100:00100000:1.0:1789762803.200504:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074658560 00000100:00100000:0.0:1789762803.200510:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762803.200525:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074658688:0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.200527:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074657920 00000100:00100000:1.0:1789762803.200528:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074658048 00000100:00100000:1.0:1789762803.200532:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074658048 00000100:00100000:2.0:1789762803.200532:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074657920 00000100:00100000:0.0:1789762803.200559:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074657792:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762803.200563:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762803.200570:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:2.0:1789762803.200570:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:2.0:1789762803.200583:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074657920:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762803.200587:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074658176:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762803.200591:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074659072:0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.200602:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074658304 00002000:00000024:1.0:1789762803.200603:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762803.200610:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074658304 00000100:00100000:3.0:1789762803.200628:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762803.200646:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074657920:12345-0@lo:400:kworker.0 Request processed in 65us (489us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762803.200653:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074658944 00000100:00100000:0.0:1789762803.200657:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074658944 00000100:00100000:3.0:1789762803.200658:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074659584:0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.200659:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00100000:1.0:1789762803.200660:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074658176:12345-0@lo:13:osp-pre-0-1.0 Request processed in 72us (338us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762803.200671:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00100000:0.0:1789762803.200683:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074659328 00000100:00100000:0.0:1789762803.200687:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074659328 00000100:00100000:1.0:1789762803.200696:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:3.0:1789762803.200696:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762803.200705:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074658688 00000100:00100000:0.0:1789762803.200709:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074658688 00000100:00100000:1.0:1789762803.200712:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074658560:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762803.200714:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1789762803.200726:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074658048:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762803.200731:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074659072 00000100:00100000:0.0:1789762803.200734:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074659072 00002000:00000024:1.0:1789762803.200736:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00002000:00000024:2.0:1789762803.200737:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762803.200758:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 362 00000100:00100000:3.0:1789762803.200765:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 363 00000100:00100000:0.0:1789762803.200770:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9fff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074658304:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762803.200784:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074658560:12345-0@lo:13:osp-pre-1-1.0 Request processed in 72us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762803.200784:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074658944:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.200785:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074658048:12345-0@lo:13:osp-pre-1-0.0 Request processed in 63us (295us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762803.200798:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1789762803.200808:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00100000:0.0:1789762803.200837:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9fff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074658304:12345-0@lo:400:kworker.0 Request processed in 52us (569us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762803.200845:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 362 00000100:00100000:2.0:1789762803.200852:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074658432 00000100:00100000:3.0:1789762803.200852:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074658944:12345-0@lo:400:kworker.0 Request processed in 75us (498us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762803.200853:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:2.0:1789762803.200859:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074658432 00000100:00100000:0.0:1789762803.200864:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:1.0:1789762803.200869:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074659328:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762803.200875:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 363 00000100:00100000:0.0:1789762803.200877:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074658688:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.200898:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074658816 00000100:00100000:3.0:1789762803.200899:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:2.0:1789762803.200906:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074658816 00000100:00100000:3.0:1789762803.200911:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074659072:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.200946:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074658688:12345-0@lo:400:kworker.0 Request processed in 74us (387us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762803.200954:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00100000:1.0:1789762803.200954:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074659328:12345-0@lo:400:kworker.0 Request processed in 87us (517us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762803.200965:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 239 00000100:00100000:1.0:1789762803.200970:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:3.0:1789762803.200970:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074659072:12345-0@lo:400:kworker.0 Request processed in 59us (339us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762803.200980:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074658432:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762803.200984:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000100:00100000:0.0:1789762803.200991:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 240 00000100:00100000:0.0:1789762803.201001:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074658816:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762803.201005:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074659456 00000100:00100000:1.0:1789762803.201011:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074659456 00000100:00100000:3.0:1789762803.201026:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d252ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074658048:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762803.201044:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074658688:0@lo:400:kworker.0 00000100:00100000:1.0:1789762803.201049:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074659840 00000100:00100000:2.0:1789762803.201051:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074658432:12345-0@lo:400:kworker.0 Request processed in 71us (780us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762803.201054:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074659840 00000100:00100000:0.0:1789762803.201059:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074658816:12345-0@lo:400:kworker.0 Request processed in 57us (702us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762803.201061:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074659072:0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.201064:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 239 00000100:00100000:0.0:1789762803.201078:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 240 00000100:00100000:1.0:1789762803.201079:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074659584 00000100:00100000:1.0:1789762803.201086:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074659584 00000100:00100000:2.0:1789762803.201109:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 140 00000100:00100000:2.0:1789762803.201119:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074659456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762803.201120:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 141 00000100:00100000:3.0:1789762803.201125:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074657920:0@lo:400:kworker.0 00000100:00080000:0.0:1789762803.201131:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762803.201132:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074659840:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762803.201137:0:3424:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967377 gen 2 00000100:00080000:0.0:1789762803.201139:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00080000:3.0:1789762803.201142:0:3424:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967376 committed transno 4294967377 00000100:00080000:0.0:1789762803.201145:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762803.201151:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00080000:3.0:1789762803.201159:0:3424:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967377 committed transno 4294967377 00000100:00100000:2.0:1789762803.201169:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074659456:12345-0@lo:400:kworker.0 Request processed in 49us (693us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762803.201170:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762803.201173:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074659840:12345-0@lo:400:kworker.0 Request processed in 45us (662us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762803.201180:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074658432:0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.201181:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 140 00000100:00100000:1.0:1789762803.201194:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 141 00000100:00100000:0.0:1789762803.201196:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074659712:0@lo:400:kworker.0 00000100:00100000:3.0:1789762803.201209:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074658944:0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.201210:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:2.0:1789762803.201225:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074659584:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.201226:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762803.201235:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074659200 00000100:00100000:1.0:1789762803.201240:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074659200 00000100:00100000:3.0:1789762803.201251:0:3424:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294967673 gen 2 00000004:00080000:3.0:1789762803.201255:0:3424:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37ff8aaa00, transno 4294967673 00000100:00100000:3.0:1789762803.201270:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074659456:0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.201279:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff70ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074658176:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762803.201286:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 367 00000100:00100000:2.0:1789762803.201287:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074659584:12345-0@lo:400:kworker.0 Request processed in 63us (591us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762803.201295:0:3426:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294967363 gen 2 00000100:00100000:2.0:1789762803.201295:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00100000:1.0:1789762803.201297:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074659200:12345-0@lo:400:kworker.0 00000004:00080000:0.0:1789762803.201302:0:3426:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37c96b4700, transno 4294967363 00000100:00100000:2.0:1789762803.201318:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074659712 00000100:00100000:3.0:1789762803.201318:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074659584:0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.201322:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff70d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074658560:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762803.201326:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074659712 00000100:00100000:1.0:1789762803.201343:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074659200:12345-0@lo:400:kworker.0 Request processed in 48us (176us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762803.201352:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 367 00000100:00100000:0.0:1789762803.201355:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074659200:0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.201355:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 143 00000100:00100000:2.0:1789762803.201368:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074659712:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762803.201394:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762803.201420:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074659712:12345-0@lo:400:kworker.0 Request processed in 54us (196us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762803.201429:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074659712:0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.201442:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 143 00000100:00100000:0.0:1789762803.201467:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074658816:0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.201475:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8ab100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074659328:0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.201481:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074659840:0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.201486:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074658304:0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.249658:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276496647369 time 276496724297 state 0x0 ref 1 00000100:00100000:0.0:1789762803.249691:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067351808:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762803.249722:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754be lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.249737:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762803.249740:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762803.249745:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754be lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.249756:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754be lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.252523:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754be lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.252533:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754be lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.252539:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754be lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.252544:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762803.252548:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754be lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.252554:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754be lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.252576:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754be lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.252582:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754be lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762803.252625:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067351808:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2940us (96586us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762803.252631:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 206 00000100:00100000:0.0:1789762803.252638:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276496647369 time 276496724297 state 0x2 ref 0 00000100:00100000:0.0:1789762803.254482:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762803.254532:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067351936 00000100:00100000:2.0:1789762803.254540:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067351936 00000100:00100000:2.0:1789762803.254575:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276496647369 time 276496724297 state 0x0 ref 1 00000100:00100000:2.0:1789762803.254585:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276596724297 time 276496724297 state 0x1 ref 1 00010000:00010000:3.0:1789762803.328056:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762803.349738:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276596724297 time 276596802666 state 0x0 ref 1 00000100:00100000:1.0:1789762803.349768:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff883480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067351936:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762803.349810:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754c5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.349840:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.349843:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.349848:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754c5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.349865:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754c5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.352627:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754c5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.352639:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754c5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.352661:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754c5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.352667:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762803.352671:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754c5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.352677:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754c5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.352691:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754c5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.352696:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754c5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.352744:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff883480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067351936:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2990us (98262us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762803.352754:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 207 00000100:00100000:1.0:1789762803.352764:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276596724297 time 276596802666 state 0x2 ref 0 00000100:00100000:0.0:1789762803.354803:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762803.354850:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067352064 00000100:00100000:3.0:1789762803.354859:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067352064 00000100:00100000:3.0:1789762803.354895:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276596724297 time 276596802666 state 0x0 ref 1 00000100:00100000:3.0:1789762803.354906:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276696802666 time 276596802666 state 0x1 ref 1 00000100:00100000:3.0:1789762803.449804:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276696802666 time 276696869629 state 0x0 ref 1 00000100:00100000:3.0:1789762803.449842:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067352064:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762803.449879:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1754cc lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.449893:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762803.449897:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762803.449901:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1754cc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.449912:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1754cc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.452902:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1754cc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.452914:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1754cc lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.452951:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1754cc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.452960:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762803.452966:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1754cc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.452977:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1754cc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.452997:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1754cc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.453006:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f1754cc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.453065:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067352064:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3245us (98262us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762803.453075:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 208 00000100:00100000:1.0:1789762803.453086:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276696802666 time 276696869629 state 0x2 ref 0 00000100:00100000:3.0:1789762803.455088:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762803.455127:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067352192 00000100:00100000:0.0:1789762803.455136:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067352192 00000100:00100000:0.0:1789762803.455174:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276696802666 time 276696869629 state 0x0 ref 1 00000100:00100000:0.0:1789762803.455184:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276796869629 time 276696869629 state 0x1 ref 1 00000100:00100000:0.0:1789762803.549881:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276796869629 time 276796946921 state 0x0 ref 1 00000100:00100000:0.0:1789762803.549908:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067352192:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762803.549993:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754d3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.550019:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762803.550022:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762803.550027:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754d3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.550038:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754d3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.552949:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754d3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.552965:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754d3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.552981:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754d3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.552988:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762803.552994:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754d3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.553004:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754d3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.553021:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754d3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.553044:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032f2d923/0xdf5a7ac59f1754d3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762803.553103:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067352192:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3204us (98014us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762803.553114:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 209 00000100:00100000:0.0:1789762803.553125:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276796869629 time 276796946921 state 0x2 ref 0 00000100:00100000:3.0:1789762803.555153:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762803.555202:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067352320 00000100:00100000:1.0:1789762803.555210:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067352320 00000100:00100000:1.0:1789762803.555248:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276796869629 time 276796946921 state 0x0 ref 1 00000100:00100000:1.0:1789762803.555268:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276896946921 time 276796946921 state 0x1 ref 1 00000100:00100000:2.0:1789762803.649982:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276896946921 time 276897042072 state 0x0 ref 1 00000100:00100000:2.0:1789762803.650012:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9af100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067352320:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762803.650051:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000394c2f2e/0xdf5a7ac59f1754da lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.650065:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762803.650068:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762803.650073:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000394c2f2e/0xdf5a7ac59f1754da lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.650084:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000394c2f2e/0xdf5a7ac59f1754da lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.652643:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000394c2f2e/0xdf5a7ac59f1754da lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.652655:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000394c2f2e/0xdf5a7ac59f1754da lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.652662:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000394c2f2e/0xdf5a7ac59f1754da lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.652668:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d925c000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762803.652672:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000394c2f2e/0xdf5a7ac59f1754da lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.652678:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000394c2f2e/0xdf5a7ac59f1754da lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.652692:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000394c2f2e/0xdf5a7ac59f1754da lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.652697:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000394c2f2e/0xdf5a7ac59f1754da lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.652753:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9af100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067352320:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2754us (97601us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762803.652775:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 210 00000100:00100000:1.0:1789762803.652784:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276896946921 time 276897042072 state 0x2 ref 0 00000100:00100000:3.0:1789762803.654653:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762803.654722:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067352448 00000100:00100000:1.0:1789762803.654730:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067352448 00000100:00100000:1.0:1789762803.654764:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276896946921 time 276897042072 state 0x0 ref 1 00000100:00100000:1.0:1789762803.654775:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276997042072 time 276897042072 state 0x1 ref 1 00000100:00100000:2.0:1789762803.750081:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276997042072 time 276997146674 state 0x0 ref 1 00000100:00100000:2.0:1789762803.750109:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067352448:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762803.750166:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007c61a906/0xdf5a7ac59f1754e1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.750179:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762803.750183:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762803.750187:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007c61a906/0xdf5a7ac59f1754e1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.750198:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007c61a906/0xdf5a7ac59f1754e1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.753046:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007c61a906/0xdf5a7ac59f1754e1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.753058:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007c61a906/0xdf5a7ac59f1754e1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.753065:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007c61a906/0xdf5a7ac59f1754e1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.753071:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d925de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762803.753075:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007c61a906/0xdf5a7ac59f1754e1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.753081:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007c61a906/0xdf5a7ac59f1754e1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.753107:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007c61a906/0xdf5a7ac59f1754e1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.753113:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007c61a906/0xdf5a7ac59f1754e1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762803.753159:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067352448:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3063us (98506us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762803.753166:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 211 00000100:00100000:2.0:1789762803.753174:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276997042072 time 276997146674 state 0x2 ref 0 00000100:00100000:0.0:1789762803.755055:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762803.755086:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067352576 00000100:00100000:2.0:1789762803.755094:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067352576 00000100:00100000:2.0:1789762803.755127:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 276997042072 time 276997146674 state 0x0 ref 1 00000100:00100000:2.0:1789762803.755138:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277097146674 time 276997146674 state 0x1 ref 1 00000100:00100000:3.0:1789762803.850180:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277097146674 time 277097244842 state 0x0 ref 1 00000100:00100000:3.0:1789762803.850206:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067352576:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762803.850243:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754e8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.850257:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762803.850260:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762803.850265:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754e8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.850283:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754e8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.853100:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754e8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.853109:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754e8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.853115:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754e8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.853135:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762803.853139:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754e8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.853146:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754e8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.853161:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754e8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762803.853166:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754e8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762803.853218:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067352576:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3022us (98164us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762803.853224:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 212 00000100:00100000:3.0:1789762803.853232:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277097146674 time 277097244842 state 0x2 ref 0 00000100:00100000:2.0:1789762803.855105:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762803.855164:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067352704 00000100:00100000:1.0:1789762803.855173:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067352704 00000100:00100000:1.0:1789762803.855218:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277097146674 time 277097244842 state 0x0 ref 1 00000100:00100000:1.0:1789762803.855231:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277197244842 time 277097244842 state 0x1 ref 1 00000100:00100000:0.0:1789762803.950281:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277197244842 time 277197340706 state 0x0 ref 1 00000100:00100000:0.0:1789762803.950313:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067352704:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762803.950352:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f1754ef lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.950365:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762803.950369:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762803.950373:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f1754ef lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762803.950385:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f1754ef lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.954352:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f1754ef lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.954363:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f1754ef lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.954372:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f1754ef lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.954394:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f1400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762803.954400:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f1754ef lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.954412:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f1754ef lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.954429:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f1754ef lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.954435:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000094cde2f/0xdf5a7ac59f1754ef lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762803.954487:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067352704:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4185us (99381us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762803.954497:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 213 00000100:00100000:2.0:1789762803.954508:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277197244842 time 277197340706 state 0x2 ref 0 00000100:00100000:2.0:1789762803.956380:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762803.956430:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067352832 00000100:00100000:0.0:1789762803.956443:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067352832 00000100:00100000:0.0:1789762803.956480:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277197244842 time 277197340706 state 0x0 ref 1 00000100:00100000:0.0:1789762803.956491:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277297340706 time 277197340706 state 0x1 ref 1 00000100:00100000:1.0:1789762804.050396:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277297340706 time 277297460134 state 0x0 ref 1 00000100:00100000:1.0:1789762804.050421:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067352832:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762804.050459:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754f6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.050473:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.050476:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.050481:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754f6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.050506:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754f6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.053362:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754f6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.053373:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754f6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.053380:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754f6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.053386:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762804.053390:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754f6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.053396:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754f6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.053411:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754f6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.053416:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f1754f6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.053474:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067352832:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3063us (97095us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762804.053482:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 214 00000100:00100000:1.0:1789762804.053490:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277297340706 time 277297460134 state 0x2 ref 0 00000100:00100000:0.0:1789762804.055421:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762804.055462:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067352960 00000100:00100000:3.0:1789762804.055470:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067352960 00000100:00100000:3.0:1789762804.055507:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277297340706 time 277297460134 state 0x0 ref 1 00000100:00100000:3.0:1789762804.055517:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277397460134 time 277297460134 state 0x1 ref 1 00000100:00100000:3.0:1789762804.150454:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277397460134 time 277397517836 state 0x0 ref 1 00000100:00100000:3.0:1789762804.150493:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067352960:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762804.150530:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754fd lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.150544:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762804.150548:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762804.150552:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754fd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.150564:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754fd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.153666:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754fd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.153677:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754fd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.153682:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754fd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.153687:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762804.153691:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754fd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.153697:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754fd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.153719:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754fd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.153725:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f1754fd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762804.153778:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067352960:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3293us (98357us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762804.153784:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 215 00000100:00100000:3.0:1789762804.153791:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277397460134 time 277397517836 state 0x2 ref 0 00000100:00100000:0.0:1789762804.155764:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762804.155833:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067353088 00000100:00100000:2.0:1789762804.155841:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067353088 00000100:00100000:2.0:1789762804.155878:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277397460134 time 277397517836 state 0x0 ref 1 00000100:00100000:2.0:1789762804.155889:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277497517836 time 277397517836 state 0x1 ref 1 00000100:00100000:2.0:1789762804.250517:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277497517836 time 277497582141 state 0x0 ref 1 00000100:00100000:2.0:1789762804.250540:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067353088:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762804.250575:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175504 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.250589:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762804.250593:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762804.250597:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175504 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.250608:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175504 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.253576:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175504 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.253588:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175504 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.253594:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175504 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.253600:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762804.253604:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175504 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.253619:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175504 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.253639:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175504 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.253647:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008731dc28/0xdf5a7ac59f175504 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762804.253704:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067353088:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3172us (97939us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762804.253715:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 216 00000100:00100000:2.0:1789762804.253723:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277497517836 time 277497582141 state 0x2 ref 0 00000100:00100000:2.0:1789762804.255597:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762804.255653:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067353216 00000100:00100000:1.0:1789762804.255661:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067353216 00000100:00100000:1.0:1789762804.255697:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277497517836 time 277497582141 state 0x0 ref 1 00000100:00100000:1.0:1789762804.255709:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277597582141 time 277497582141 state 0x1 ref 1 00000100:00100000:1.0:1789762804.350573:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277597582141 time 277597638286 state 0x0 ref 1 00000100:00100000:1.0:1789762804.350599:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067353216:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762804.350637:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17550b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.350650:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.350654:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.350658:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17550b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.350669:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17550b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.352115:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762804.353408:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17550b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.353419:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17550b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.353426:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17550b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.353430:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762804.353434:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17550b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.353439:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17550b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.353459:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17550b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.353464:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17550b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.353510:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067353216:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2922us (97913us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762804.353517:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 217 00000100:00100000:1.0:1789762804.353536:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277597582141 time 277597638286 state 0x2 ref 0 00000100:00100000:2.0:1789762804.355409:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762804.355450:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067353344 00000100:00100000:3.0:1789762804.355459:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067353344 00000100:00100000:3.0:1789762804.355496:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277597582141 time 277597638286 state 0x0 ref 1 00000100:00100000:3.0:1789762804.355506:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277697638286 time 277597638286 state 0x1 ref 1 00000100:00100000:0.0:1789762804.450656:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277697638286 time 277697721106 state 0x0 ref 1 00000100:00100000:0.0:1789762804.450680:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067353344:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762804.450718:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175512 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.450732:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762804.450735:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762804.450739:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175512 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.450751:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175512 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.453720:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175512 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.453731:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175512 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.453739:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175512 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.453756:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d92a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762804.453760:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175512 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.453767:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175512 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.453781:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175512 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.453786:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175512 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762804.453845:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067353344:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3173us (98437us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762804.453852:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 218 00000100:00100000:2.0:1789762804.453860:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277697638286 time 277697721106 state 0x2 ref 0 00000100:00100000:2.0:1789762804.456168:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762804.456244:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067353472 00000100:00100000:0.0:1789762804.456254:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067353472 00000100:00100000:0.0:1789762804.456307:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277697638286 time 277697721106 state 0x0 ref 1 00000100:00100000:0.0:1789762804.456324:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277797721106 time 277697721106 state 0x1 ref 1 00000100:00100000:3.0:1789762804.550743:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277797721106 time 277797808169 state 0x0 ref 1 00000100:00100000:3.0:1789762804.550769:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067353472:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762804.550807:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175519 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.550853:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762804.550856:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762804.550861:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175519 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.550872:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175519 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.553715:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175519 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.553724:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175519 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.553731:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175519 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.553736:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762804.553739:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175519 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.553745:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175519 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.553758:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175519 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.553773:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175519 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762804.553843:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067353472:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3084us (97678us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762804.553850:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 219 00000100:00100000:3.0:1789762804.553858:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277797721106 time 277797808169 state 0x2 ref 0 00000100:00100000:0.0:1789762804.555670:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762804.555725:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067353600 00000100:00100000:1.0:1789762804.555733:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067353600 00000100:00100000:1.0:1789762804.555772:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277797721106 time 277797808169 state 0x0 ref 1 00000100:00100000:1.0:1789762804.555783:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277897808169 time 277797808169 state 0x1 ref 1 00000100:00100000:2.0:1789762804.650857:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277897808169 time 277897911116 state 0x0 ref 1 00000100:00100000:2.0:1789762804.650887:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067353600:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762804.650950:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000dfeb827/0xdf5a7ac59f175520 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.650981:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762804.650984:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762804.650990:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000dfeb827/0xdf5a7ac59f175520 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762804.651001:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000dfeb827/0xdf5a7ac59f175520 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.653892:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000dfeb827/0xdf5a7ac59f175520 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.653904:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000dfeb827/0xdf5a7ac59f175520 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.653911:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000dfeb827/0xdf5a7ac59f175520 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.653941:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d17db800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762804.653945:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000dfeb827/0xdf5a7ac59f175520 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.653953:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000dfeb827/0xdf5a7ac59f175520 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.653968:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000dfeb827/0xdf5a7ac59f175520 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.653973:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000dfeb827/0xdf5a7ac59f175520 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762804.654019:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067353600:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3143us (98349us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762804.654027:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 220 00000100:00100000:0.0:1789762804.654043:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277897808169 time 277897911116 state 0x2 ref 0 00000100:00100000:0.0:1789762804.655954:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762804.656005:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067353728 00000100:00100000:2.0:1789762804.656014:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067353728 00000100:00100000:2.0:1789762804.656057:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277897808169 time 277897911116 state 0x0 ref 1 00000100:00100000:2.0:1789762804.656070:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277997911116 time 277897911116 state 0x1 ref 1 00000100:00100000:1.0:1789762804.750957:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277997911116 time 277997993934 state 0x0 ref 1 00000100:00100000:1.0:1789762804.750983:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067353728:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762804.751021:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175527 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.751039:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.751044:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.751051:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175527 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.751068:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175527 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.753741:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175527 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.753752:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175527 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.753770:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175527 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.753775:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762804.753778:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175527 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.753784:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175527 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.753799:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175527 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.753823:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175527 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.753872:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067353728:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2899us (97919us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762804.753879:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 221 00000100:00100000:1.0:1789762804.753887:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277997911116 time 277997993934 state 0x2 ref 0 00000100:00100000:0.0:1789762804.755789:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762804.755845:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067353856 00000100:00100000:3.0:1789762804.755854:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067353856 00000100:00100000:3.0:1789762804.755897:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 277997911116 time 277997993934 state 0x0 ref 1 00000100:00100000:3.0:1789762804.755907:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278097993934 time 277997993934 state 0x1 ref 1 00000100:00100000:3.0:1789762804.851023:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278097993934 time 278098083191 state 0x0 ref 1 00000100:00100000:3.0:1789762804.851055:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067353856:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762804.851097:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17552e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.851114:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762804.851118:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762804.851124:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17552e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.851156:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17552e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.855243:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17552e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.855257:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17552e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.855267:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17552e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.855275:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d92600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762804.855280:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17552e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.855290:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17552e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.855308:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17552e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.855316:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17552e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762804.855394:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067353856:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4351us (99604us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762804.855405:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 222 00000100:00100000:3.0:1789762804.855416:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278097993934 time 278098083191 state 0x2 ref 0 00000100:00100000:2.0:1789762804.857413:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762804.857463:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067353984 00000100:00100000:0.0:1789762804.857472:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067353984 00000100:00100000:0.0:1789762804.857510:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278097993934 time 278098083191 state 0x0 ref 1 00000100:00100000:0.0:1789762804.857521:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278198083191 time 278098083191 state 0x1 ref 1 00000100:00100000:0.0:1789762804.951092:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278198083191 time 278198156839 state 0x0 ref 1 00000100:00100000:0.0:1789762804.951118:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067353984:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762804.951155:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000456b72f8/0xdf5a7ac59f175535 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.951169:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762804.951172:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762804.951177:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000456b72f8/0xdf5a7ac59f175535 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.951188:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000456b72f8/0xdf5a7ac59f175535 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.953880:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000456b72f8/0xdf5a7ac59f175535 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.953908:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000456b72f8/0xdf5a7ac59f175535 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.953915:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000456b72f8/0xdf5a7ac59f175535 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.953939:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9210c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762804.953943:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000456b72f8/0xdf5a7ac59f175535 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.953950:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000456b72f8/0xdf5a7ac59f175535 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.953964:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000456b72f8/0xdf5a7ac59f175535 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762804.953969:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000456b72f8/0xdf5a7ac59f175535 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762804.954015:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067353984:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2908us (96603us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762804.954021:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 223 00000100:00100000:0.0:1789762804.954029:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278198083191 time 278198156839 state 0x2 ref 0 00000100:00100000:0.0:1789762804.955995:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762804.956062:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067354112 00000100:00100000:1.0:1789762804.956071:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067354112 00000100:00100000:1.0:1789762804.956109:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278198083191 time 278198156839 state 0x0 ref 1 00000100:00100000:1.0:1789762804.956129:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278298156839 time 278198156839 state 0x1 ref 1 00000100:00100000:1.0:1789762805.051158:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278298156839 time 278298219053 state 0x0 ref 1 00000100:00100000:1.0:1789762805.051198:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067354112:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762805.051240:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17553c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789762805.051246:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762805.051259:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.051265:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.051272:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17553c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.051286:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17553c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.055523:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17553c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.055540:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17553c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.055552:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17553c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.055579:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762805.055585:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17553c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.055595:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17553c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.055615:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17553c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.055625:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f17553c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.055687:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067354112:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4510us (99692us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762805.055698:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 224 00000100:00100000:1.0:1789762805.055710:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278298156839 time 278298219053 state 0x2 ref 0 00000100:00100000:0.0:1789762805.057726:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762805.057772:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067354240 00000100:00100000:3.0:1789762805.057781:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067354240 00000100:00100000:3.0:1789762805.057837:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278298156839 time 278298219053 state 0x0 ref 1 00000100:00100000:3.0:1789762805.057852:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278398219053 time 278298219053 state 0x1 ref 1 00000100:00100000:2.0:1789762805.151261:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278398219053 time 278398326496 state 0x0 ref 1 00000100:00100000:2.0:1789762805.151288:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067354240:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762805.151322:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f175543 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.151336:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762805.151339:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762805.151344:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f175543 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.151372:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f175543 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.154630:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f175543 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.154642:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f175543 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.154649:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f175543 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.154654:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795a600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762805.154658:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f175543 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.154664:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f175543 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.154677:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f175543 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.154682:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f175543 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762805.154741:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067354240:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3463us (97014us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762805.154752:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 225 00000100:00100000:2.0:1789762805.154763:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278398219053 time 278398326496 state 0x2 ref 0 00000100:00100000:2.0:1789762805.156963:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762805.157012:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067354368 00000100:00100000:0.0:1789762805.157020:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067354368 00000100:00100000:0.0:1789762805.157056:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278398219053 time 278398326496 state 0x0 ref 1 00000100:00100000:0.0:1789762805.157067:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278498326496 time 278398326496 state 0x1 ref 1 00000100:00100000:2.0:1789762805.248089:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074659968:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762805.248160:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762805.248219:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074659968 00000100:00100000:0.0:1789762805.248228:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074659968 00000100:00100000:0.0:1789762805.248282:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:0.0:1789762805.248295:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074659968:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762805.248328:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762805.248381:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074659968:12345-0@lo:41:osp-pre-1-0.0 Request processed in 88us (219us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762805.248390:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00100000:2.0:1789762805.248429:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074659968:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762805.251315:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278498326496 time 278498382431 state 0x0 ref 1 00000100:00100000:3.0:1789762805.251332:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067354368:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762805.251363:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17554a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.251379:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762805.251383:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762805.251389:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17554a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.251403:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17554a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.254757:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17554a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.254770:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17554a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.254780:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17554a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.254788:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d92600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762805.254793:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17554a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.254802:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17554a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.254835:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17554a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.254844:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17554a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762805.254900:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067354368:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3570us (97937us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762805.254909:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 226 00000100:00100000:3.0:1789762805.254967:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278498326496 time 278498382431 state 0x2 ref 0 00000100:00100000:0.0:1789762805.256963:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762805.257012:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067354496 00000100:00100000:1.0:1789762805.257020:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067354496 00000100:00100000:1.0:1789762805.257055:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278498326496 time 278498382431 state 0x0 ref 1 00000100:00100000:1.0:1789762805.257066:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278598382431 time 278498382431 state 0x1 ref 1 00000100:00100000:0.0:1789762805.351396:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278598382431 time 278598455124 state 0x0 ref 1 00000100:00100000:0.0:1789762805.351429:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067354496:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762805.351477:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007c73b89d/0xdf5a7ac59f175551 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.351490:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762805.351494:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762805.351499:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007c73b89d/0xdf5a7ac59f175551 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.351522:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007c73b89d/0xdf5a7ac59f175551 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.354628:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007c73b89d/0xdf5a7ac59f175551 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.354643:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007c73b89d/0xdf5a7ac59f175551 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.354654:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007c73b89d/0xdf5a7ac59f175551 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.354663:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5871c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762805.354668:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007c73b89d/0xdf5a7ac59f175551 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.354679:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007c73b89d/0xdf5a7ac59f175551 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.354698:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007c73b89d/0xdf5a7ac59f175551 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.354709:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007c73b89d/0xdf5a7ac59f175551 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762805.354786:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067354496:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3369us (97822us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762805.354798:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 227 00000100:00100000:0.0:1789762805.354824:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278598382431 time 278598455124 state 0x2 ref 0 00000100:00100000:0.0:1789762805.356895:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762805.356943:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067354624 00000100:00100000:2.0:1789762805.356951:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067354624 00000100:00100000:2.0:1789762805.356987:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278598382431 time 278598455124 state 0x0 ref 1 00000100:00100000:2.0:1789762805.356998:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278698455124 time 278598455124 state 0x1 ref 1 00010000:00010000:1.0:1789762805.376041:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762805.451476:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278698455124 time 278698541269 state 0x0 ref 1 00000100:00100000:1.0:1789762805.451502:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067354624:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762805.451539:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175558 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.451552:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.451556:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.451561:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175558 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.451572:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175558 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.454535:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175558 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.454546:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175558 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.454552:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175558 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.454558:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762805.454562:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175558 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.454580:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175558 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.454593:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175558 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.454599:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175558 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.454646:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067354624:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3156us (97752us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762805.454653:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 228 00000100:00100000:1.0:1789762805.454661:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278698455124 time 278698541269 state 0x2 ref 0 00000100:00100000:3.0:1789762805.457249:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762805.457318:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067354752 00000100:00100000:0.0:1789762805.457328:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067354752 00000100:00100000:0.0:1789762805.457382:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278698455124 time 278698541269 state 0x0 ref 1 00000100:00100000:0.0:1789762805.457398:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278798541269 time 278698541269 state 0x1 ref 1 00000100:00100000:3.0:1789762805.551628:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278798541269 time 278798693604 state 0x0 ref 1 00000100:00100000:3.0:1789762805.551654:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067354752:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762805.551692:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17555f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.551706:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762805.551710:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762805.551714:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17555f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.551725:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17555f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.554634:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17555f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.554645:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17555f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.554651:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17555f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.554673:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d92600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762805.554677:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17555f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.554684:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17555f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.554697:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17555f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.554702:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000099e11113/0xdf5a7ac59f17555f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762805.554754:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067354752:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3111us (97512us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762805.554761:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 229 00000100:00100000:3.0:1789762805.554769:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278798541269 time 278798693604 state 0x2 ref 0 00000100:00100000:3.0:1789762805.556887:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762805.556941:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067354880 00000100:00100000:0.0:1789762805.556949:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067354880 00000100:00100000:0.0:1789762805.556990:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278798541269 time 278798693604 state 0x0 ref 1 00000100:00100000:0.0:1789762805.557001:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278898693604 time 278798693604 state 0x1 ref 1 00000100:00100000:0.0:1789762805.651688:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278898693604 time 278898753179 state 0x0 ref 1 00000100:00100000:0.0:1789762805.651714:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067354880:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762805.651753:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175566 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.651785:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762805.651788:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762805.651793:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175566 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.651816:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175566 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.654821:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175566 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.654834:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175566 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.654844:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175566 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.654865:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762805.654870:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175566 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.654881:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175566 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.654898:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175566 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.654907:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175566 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762805.655000:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067354880:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3294us (98112us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762805.655010:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 230 00000100:00100000:2.0:1789762805.655021:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278898693604 time 278898753179 state 0x2 ref 0 00000100:00100000:3.0:1789762805.679797:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762805.679876:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067355008 00000100:00100000:1.0:1789762805.679885:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067355008 00000100:00100000:1.0:1789762805.679955:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 31 00000100:00100000:1.0:1789762805.679964:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9afb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:6357:x1876702067355008:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762805.679977:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762805.680013:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052f30ec7/0xdf5a7ac59f17529c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c687 expref: 6 pid: 9037 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.680034:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762805.680081:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9afb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:6357:x1876702067355008:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 119us (292us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762805.680099:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 31 00000100:00100000:0.0:1789762805.691339:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762805.691395:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067355136 00000100:00100000:3.0:1789762805.691405:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067355136 00000100:00080000:0.0:1789762805.691451:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762805.691468:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 199 00000100:00100000:3.0:1789762805.691479:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:13087:x1876702067355136:12345-192.168.203.37@tcp:101:bash.0 00010000:00010000:3.0:1789762805.691507:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762805.691541:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92600/0xdf5a7ac59f17556d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c68e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.691593:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93200/0xdf5a7ac59f175574 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.691604:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93200/0xdf5a7ac59f175574 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.691612:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93200/0xdf5a7ac59f175574 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762805.691641:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 16384 00000004:00200000:3.0:1789762805.691645:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762805.691662:0:6419:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93200/0xdf5a7ac59f175574 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.691687:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92600/0xdf5a7ac59f17556d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c68e expref: 11 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.691703:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93200/0xdf5a7ac59f175574 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c68e expref: 10 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.691711:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:3.0:1789762805.691714:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762805.691717:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d93200, rc 0) 00000100:00100000:3.0:1789762805.691766:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:13087:x1876702067355136:12345-192.168.203.37@tcp:101:bash.0 Request processed in 289us (432us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762805.691773:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 199 00000100:00100000:0.0:1789762805.695673:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762805.695732:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067355392 00000100:00100000:2.0:1789762805.695737:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067355392 00000100:00100000:2.0:1789762805.695772:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 200 00000100:00100000:2.0:1789762805.695781:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:13414:x1876702067355392:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:2.0:1789762805.695790:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762805.695813:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afc00/0xdf5a7ac59f17557b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c695 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762805.695877:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 16384 00010000:00010000:2.0:1789762805.696049:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afc00/0xdf5a7ac59f17557b lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c695 expref: 12 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.696062:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762805.696065:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762805.696069:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afc00/0xdf5a7ac59f17557b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c695 expref: 12 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.696080:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d10afc00, rc 0) 00000100:00100000:2.0:1789762805.696113:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:13414:x1876702067355392:12345-192.168.203.37@tcp:101:rm.0 Request processed in 333us (441us total) trans 4294967378 rc 301/301 00000100:00100000:2.0:1789762805.696121:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 200 00000100:00100000:2.0:1789762805.698047:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762805.698101:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067355520 00000100:00100000:0.0:1789762805.698107:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067355520 00000100:00100000:0.0:1789762805.698138:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 50 00000100:00100000:0.0:1789762805.698145:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:13414:x1876702067355520:12345-192.168.203.37@tcp:37:rm.0 00000100:00100000:1.0:1789762805.700388:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:13414:x1876702067355520:12345-192.168.203.37@tcp:37:rm.0 Request processed in 2242us (2340us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762805.700396:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 50 00000100:00100000:2.0:1789762805.701752:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762805.701799:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067355776 00000100:00100000:0.0:1789762805.701822:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067355776 00000100:00100000:0.0:1789762805.701851:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 51 00000100:00100000:0.0:1789762805.701857:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:13414:x1876702067355776:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:0.0:1789762805.701980:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:13414:x1876702067355776:12345-192.168.203.37@tcp:35:rm.0 Request processed in 122us (228us total) trans 4294967379 rc 0/0 00000100:00100000:0.0:1789762805.701987:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 51 00000100:00100000:2.0:1789762805.702897:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762805.702944:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067356032 00000100:00100000:0.0:1789762805.702949:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067356032 00000100:00100000:0.0:1789762805.702975:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 201 00000100:00100000:0.0:1789762805.702987:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:13414:x1876702067356032:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:0.0:1789762805.702997:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762805.703006:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f175589 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c69c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.703044:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f175590 lrc: 3/1,0 mode: --/CR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.703053:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f175590 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0xe:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.703060:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f175590 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0xe:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762805.703066:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 16384 00010000:00010000:0.0:1789762805.703108:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94800/0xdf5a7ac59f175589 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c69c expref: 15 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.703118:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f175590 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xe:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c69c expref: 14 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.703123:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:0.0:1789762805.703125:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1789762805.703127:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a95c00, rc 0) 00000100:00100000:0.0:1789762805.703153:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13414:x1876702067356032:12345-192.168.203.37@tcp:101:rm.0 Request processed in 167us (259us total) trans 4294967380 rc 0/0 00000100:00100000:0.0:1789762805.703158:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 201 00000100:00100000:2.0:1789762805.704204:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762805.704240:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067356160 00000100:00100000:0.0:1789762805.704245:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067356160 00000100:00100000:0.0:1789762805.704276:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 202 00000100:00100000:0.0:1789762805.704282:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13414:x1876702067356160:12345-192.168.203.37@tcp:41:rm.0 00000004:00000024:0.0:1789762805.704314:0:13389:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7614758912 avail 7603224576 00000100:00100000:0.0:1789762805.704346:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13414:x1876702067356160:12345-192.168.203.37@tcp:41:rm.0 Request processed in 64us (143us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762805.704351:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 202 00000100:00100000:2.0:1789762805.705060:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762805.705102:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067356416 00000100:00100000:1.0:1789762805.705108:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067356416 00000100:00100000:1.0:1789762805.705142:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 52 00000100:00100000:1.0:1789762805.705149:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbe850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13414:x1876702067356416:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:1.0:1789762805.705310:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbe850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:13414:x1876702067356416:12345-192.168.203.37@tcp:35:rm.0 Request processed in 161us (251us total) trans 4294967381 rc 0/0 00000100:00100000:1.0:1789762805.705316:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 52 00000100:00100000:2.0:1789762805.706075:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762805.706136:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067356672 00000100:00100000:1.0:1789762805.706144:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067356672 00000100:00100000:1.0:1789762805.706189:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 203 00000100:00100000:1.0:1789762805.706202:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:13414:x1876702067356672:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:1.0:1789762805.706239:0:6420:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1789762805.706260:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37e1b10200/0xdf5a7ac59f17528e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x10:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c680 expref: 15 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.706280:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934600/0xdf5a7ac59f174faf lrc: 0/0,0 mode: --/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c679 expref: 14 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.706292:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff9c37c72bae00) 00010000:00010000:1.0:1789762805.706307:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93200/0xdf5a7ac59f175574 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c68e expref: 13 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.706336:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:1.0:1789762805.706341:0:6420:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762805.706357:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f17559e lrc: 3/0,1 mode: --/CW res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.706368:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f17559e lrc: 3/0,1 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.706378:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f17559e lrc: 3/0,1 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.706395:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755a5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xe:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.706405:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755a5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.706414:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755a5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.706754:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1755ac lrc: 3/0,1 mode: --/EX res: [0x200000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.706764:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1755ac lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.706772:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1755ac lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789762805.707606:0:6420:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].5, 64 off8512 00000004:00080000:1.0:1789762805.707614:0:6420:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 278954696029ns, next at 279954694458ns, rc = 0 00010000:00010000:1.0:1789762805.707656:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1755b3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707663:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1755b3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707669:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1755b3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707673:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1755b3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707678:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1755b3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707682:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1755b3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707699:0:6420:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762805.707709:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1755ac lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707713:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1755ac lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707718:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1755ac lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707723:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762805.707727:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1755ac lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707732:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1755ac lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.707736:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762805.707748:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1755b3 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707749:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff9c37db743700) 00010000:00010000:1.0:1789762805.707754:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762805.707758:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1755ac lrc: 2/0,0 mode: --/EX res: [0x200000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.707764:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762805.707766:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1755ac lrc: 0/0,0 mode: --/EX res: [0x200000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.707771:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1755b3 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707779:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f17559e lrc: 4/0,1 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707785:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f17559e lrc: 4/0,1 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.707817:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1755b3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707819:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f17559e lrc: 3/0,0 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707836:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f17559e lrc: 4/0,0 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707843:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762805.707844:0:8069:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff9c37db743700) 00010000:00010000:1.0:1789762805.707845:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f17559e lrc: 4/0,0 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.707851:0:8069:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789762805.707855:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f17559e lrc: 5/0,0 mode: CW/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.707857:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1755b3 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707867:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:2.0:1789762805.707868:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1755b3 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707871:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762805.707876:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f17559e lrc: 3/0,0 mode: --/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.707890:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762805.707892:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f17559e lrc: 0/0,0 mode: --/CW res: [0x200000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.707894:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762805.707897:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00010000:1.0:1789762805.707905:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000866b2a6a reply state 00000000ba381df5 transno 4294967382 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755a5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707914:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755a5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.707935:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x4009f790 (ffff9c37db743d00) 00010000:00010000:1.0:1789762805.707937:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1789762805.707939:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762805.707976:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13414:x1876702067356672:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1779us (1901us total) trans 4294967382 rc 0/0 00000100:00100000:1.0:1789762805.707981:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 203 00000100:00100000:2.0:1789762805.708967:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762805.708998:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067356928 00000100:00100000:0.0:1789762805.709003:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067356928 00000100:00100000:0.0:1789762805.709031:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 53 00000100:00100000:0.0:1789762805.709037:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13414:x1876702067356928:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:0.0:1789762805.709125:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:13414:x1876702067356928:12345-192.168.203.37@tcp:35:rm.0 Request processed in 88us (160us total) trans 4294967383 rc 0/0 00000100:00100000:0.0:1789762805.709131:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 53 00000100:00100000:2.0:1789762805.709895:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762805.709948:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067357184 00000100:00100000:1.0:1789762805.709954:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067357184 00000100:00100000:1.0:1789762805.709979:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 204 00000100:00100000:1.0:1789762805.709986:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:13414:x1876702067357184:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:1.0:1789762805.710008:0:6418:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1789762805.710018:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f175590 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xe:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c69c expref: 15 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710031:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94000/0xdf5a7ac59f174f93 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c672 expref: 14 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710038:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:1.0:1789762805.710054:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ff934e00/0xdf5a7ac59f174f31 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c65d expref: 13 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710060:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762805.710062:0:6418:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762805.710071:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1755ba lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710077:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1755ba lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710082:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1755ba lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710090:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f1755c1 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710095:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f1755c1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710100:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f1755c1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710285:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231a00/0xdf5a7ac59f1755c8 lrc: 3/0,1 mode: --/EX res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710294:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231a00/0xdf5a7ac59f1755c8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710302:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231a00/0xdf5a7ac59f1755c8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762805.710622:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 16384 00010000:00010000:1.0:1789762805.710634:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231a00/0xdf5a7ac59f1755c8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0xe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710639:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231a00/0xdf5a7ac59f1755c8 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0xe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710643:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231a00/0xdf5a7ac59f1755c8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710647:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762805.710649:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231a00/0xdf5a7ac59f1755c8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0xe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710653:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231a00/0xdf5a7ac59f1755c8 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0xe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710659:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff9c37db743200) 00010000:00010000:1.0:1789762805.710660:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762805.710662:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231a00/0xdf5a7ac59f1755c8 lrc: 2/0,0 mode: --/EX res: [0x200000402:0xe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710666:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231a00/0xdf5a7ac59f1755c8 lrc: 0/0,0 mode: --/EX res: [0x200000402:0xe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710677:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1755ba lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710681:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1755ba lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710685:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1755ba lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710689:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1755ba lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710696:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762805.710699:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1755ba lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710702:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1755ba lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710707:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762805.710708:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762805.710710:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1755ba lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710714:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1755ba lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762805.710719:0:6418:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009eb4c29d reply state 00000000052c897c transno 4294967384 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f1755c1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710724:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f1755c1 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.710728:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743a00) 00010000:00010000:1.0:1789762805.710729:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1789762805.710731:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762805.710765:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13414:x1876702067357184:12345-192.168.203.37@tcp:36:rm.0 Request processed in 779us (871us total) trans 4294967384 rc 0/0 00000100:00100000:1.0:1789762805.710770:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 204 00000100:00100000:0.0:1789762806.273083:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074660096:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789762806.273286:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762806.273378:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074660096 00000100:00100000:0.0:1789762806.273385:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074660096 00000100:00100000:0.0:1789762806.273428:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:0.0:1789762806.273438:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074660096:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762806.273468:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762806.273526:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074660096:12345-0@lo:41:osp-pre-0-1.0 Request processed in 89us (241us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762806.273536:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00100000:2.0:1789762806.273552:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074660096:0@lo:41:osp-pre-0-1.0 00000001:00080000:2.0:1789762806.285177:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967383 is committed 00010000:00010000:1.0:1789762806.285192:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755a5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762806.285206:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967384 is committed 00000004:00080000:2.0:1789762806.285221:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 7 committed 00010000:00010000:1.0:1789762806.285222:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755a5 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.285235:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755a5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.285241:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f1755c1 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.285242:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762806.285251:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755a5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.285263:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755a5 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.285271:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f1755c1 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.285278:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f1755c1 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.285283:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762806.285286:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f1755c1 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.285292:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f1755c1 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.285309:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743a00) 00010000:00010000:1.0:1789762806.285309:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x4009f790 (ffff9c37db743d00) 00010000:00010000:0.0:1789762806.285312:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762806.285317:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762806.285319:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f1755c1 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.285324:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755a5 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.285330:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f1755c1 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.285339:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755a5 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0xe:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762806.286103:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074660224:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1789762806.286156:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762806.286186:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074660224 00000100:00100000:1.0:1789762806.286195:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074660224 00000100:00100000:1.0:1789762806.286263:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00100000:1.0:1789762806.286277:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074660224:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1789762806.286303:0:8054:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:5 count 1 00002000:00010000:1.0:1789762806.286333:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x5:0x0] initial LVB size: 104857600, mtime: 0x6aad9ceb, atime: 0x0, ctime: 0x6aad9ceb, blocks: 0x3200d 00010000:00010000:1.0:1789762806.286342:0:8054:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f1755cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.286370:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=279s v=(0 0) 00010000:00010000:1.0:1789762806.286376:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.286383:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f1755cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.286400:0:8054:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f1755cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.286410:0:8054:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f1755cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.286419:0:8054:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f1755cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.286438:0:8054:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762806.286451:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762806.286458:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f1755cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762806.286467:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762806.286472:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f1755cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762806.286481:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f1755cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762806.286495:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f1755cf lrc: 1/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762806.286503:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f1755cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762806.286519:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762806.286521:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762806.286524:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762806.287119:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074660224:12345-0@lo:6:osp-syn-0-0.0 Request processed in 844us (958us total) trans 4294967674 rc 0/0 00000100:00100000:1.0:1789762806.287130:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00080000:1.0:1789762806.287159:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00080000:0.0:1789762806.287164:0:3426:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37d8c9e680/2, rc 0, transno 4294967674 00000100:00100000:0.0:1789762806.287172:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074660224:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1789762806.400018:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1789762806.581136:0:15262:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 279828215650ns, next at 279954694458ns, rc = 0 00000004:00080000:1.0:1789762806.581264:0:15262:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 279828345208ns, next at 267207136663ns, rc = 0 00000004:00080000:1.0:1789762806.581425:0:15262:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 279828506018ns, next at 0ns, rc = 0 00000004:00080000:1.0:1789762806.581564:0:15262:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 279828645273ns, next at 267207169995ns, rc = 0 00000004:00080000:2.0:1789762806.582856:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 8 committed 00000004:00080000:2.0:1789762806.584132:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 4 committed 00000004:00080000:3.0:1789762806.589345:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 11 committed 00000004:00080000:3.0:1789762806.590176:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 11 committed 00000100:00100000:2.0:1789762807.240120:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762807.240159:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762807.240172:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067357312 00000100:00100000:2.0:1789762807.240186:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762807.240187:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067357312 00000100:00100000:2.0:1789762807.240223:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762807.240227:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067357440 00000100:00100000:0.0:1789762807.240243:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067357440 00000100:00100000:1.0:1789762807.240258:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067357568 00000100:00100000:2.0:1789762807.240267:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762807.240270:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067357568 00000100:00100000:3.0:1789762807.240275:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 335 00000100:00100000:3.0:1789762807.240294:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067357312:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762807.240322:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762807.240359:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067357952 00000100:00100000:1.0:1789762807.240363:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067357952 00000100:00100000:0.0:1789762807.240370:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 368 00000100:00100000:2.0:1789762807.240380:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762807.240394:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067357568:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762807.240397:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067357312:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 109us (286us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762807.240402:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 369 00000100:00100000:1.0:1789762807.240425:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067357440:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762807.240436:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 335 00000100:00100000:0.0:1789762807.240488:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067357568:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 99us (297us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762807.240493:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067357440:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 72us (327us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762807.240502:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 368 00000100:00100000:1.0:1789762807.240510:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 369 00000100:00100000:3.0:1789762807.240512:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067358080 00000100:00100000:3.0:1789762807.240519:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067358080 00000100:00100000:0.0:1789762807.240545:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 370 00000100:00100000:0.0:1789762807.240561:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067357952:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762807.240580:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067357696 00000100:00100000:1.0:1789762807.240586:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067357696 00000100:00100000:0.0:1789762807.240621:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067357952:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 65us (300us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762807.240627:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 370 00000100:00100000:3.0:1789762807.240646:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 371 00000100:00100000:1.0:1789762807.240649:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067357824 00000100:00100000:1.0:1789762807.240653:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067357824 00000100:00100000:2.0:1789762807.240665:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 144 00000100:00100000:3.0:1789762807.240666:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067358080:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762807.240682:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067357696:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762807.240695:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 145 00000100:00100000:1.0:1789762807.240704:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067357824:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762807.240723:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067358080:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 65us (345us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762807.240728:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067357696:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 47us (501us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762807.240747:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067357824:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 35us (475us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762807.240747:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 144 00000100:00100000:1.0:1789762807.240765:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 145 00000100:00080000:2.0:1789762807.240846:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762807.240853:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762807.240857:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762807.240860:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762807.241321:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 371 00010000:00010000:1.0:1789762807.424133:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762808.320053:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074660480:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762808.320062:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074660736:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762808.320065:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074660608:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762808.320127:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762808.320135:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074660352:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762808.320162:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074660480 00000100:00100000:0.0:1789762808.320168:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074660480 00000100:00100000:1.0:1789762808.320185:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1789762808.320205:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762808.320223:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762808.320237:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074660352 00000100:00100000:0.0:1789762808.320241:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074660352 00000100:00100000:2.0:1789762808.320250:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074660608 00000100:00100000:2.0:1789762808.320256:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074660608 00000100:00100000:1.0:1789762808.320272:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:0.0:1789762808.320280:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:1.0:1789762808.320285:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074660480:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762808.320292:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074660352:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762808.320299:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 156 00000100:00100000:3.0:1789762808.320306:0:13091:0:(service.c:3462:ptlrpc_start_thread()) ost_create[0] started 4 min 2 max 16 00000100:00100000:2.0:1789762808.320310:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074660608:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789762808.320319:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762808.320319:0:13091:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_create00_004' 00002000:00000024:1.0:1789762808.320320:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00002000:00000024:2.0:1789762808.320332:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3754950656 avail 3752853504 00000100:00100000:1.0:1789762808.320398:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074660480:12345-0@lo:13:osp-pre-1-0.0 Request processed in 114us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762808.320399:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074660352:12345-0@lo:13:osp-pre-1-1.0 Request processed in 110us (215us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762808.320401:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762808.320403:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074660608:12345-0@lo:13:osp-pre-0-1.0 Request processed in 94us (177us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762808.320406:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762808.320420:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:2.0:1789762808.320430:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 156 00000100:00100000:1.0:1789762808.320437:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:0.0:1789762808.320452:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762808.320475:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074660736 00000100:00100000:1.0:1789762808.320482:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074660736 00000100:00100000:2.0:1789762808.320483:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074660608:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762808.320518:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074660352:0@lo:13:osp-pre-1-1.0 00000100:00080000:2.0:1789762808.320521:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762808.320534:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:3.0:1789762808.320536:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074660480:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762808.320543:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074660736:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789762808.320554:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3754950656 avail 3752853504 00000100:00100000:1.0:1789762808.320586:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074660736:12345-0@lo:13:osp-pre-0-0.0 Request processed in 46us (135us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762808.320590:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:0.0:1789762808.320612:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074660736:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762808.321005:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074660864:0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.321049:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762808.321098:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff685f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074661248:0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.321111:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9246300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074661376:0@lo:400:kworker.0 00000100:00100000:0.0:1789762808.321140:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762808.321150:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762808.321182:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9245180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074661760:0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.321184:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9244a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074661888:0@lo:400:kworker.0 00000100:00100000:0.0:1789762808.321231:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762808.321237:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074660864 00000100:00100000:3.0:1789762808.321241:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074660864 00000100:00100000:0.0:1789762808.321285:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074661248 00000100:00100000:3.0:1789762808.321286:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:0.0:1789762808.321293:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074661248 00000100:00100000:3.0:1789762808.321302:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074660864:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762808.321341:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074661760 00000100:00100000:0.0:1789762808.321346:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074661760 00000100:00100000:3.0:1789762808.321356:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074660864:12345-0@lo:400:kworker.0 Request processed in 58us (308us total) trans 0 rc 0/0 00000100:00080000:2.0:1789762808.321365:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762808.321371:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00100000:0.0:1789762808.321386:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:0.0:1789762808.321396:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074661248:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762808.321398:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 373 00000100:00100000:2.0:1789762808.321414:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074661760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.321432:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074661120:0@lo:400:kworker.0 00000100:00100000:0.0:1789762808.321449:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074661248:12345-0@lo:400:kworker.0 Request processed in 56us (312us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762808.321456:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00100000:1.0:1789762808.321459:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074661376 00000100:00100000:1.0:1789762808.321466:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074661376 00000100:00100000:2.0:1789762808.321472:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074661760:12345-0@lo:400:kworker.0 Request processed in 60us (240us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762808.321472:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762808.321490:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 373 00000100:00100000:0.0:1789762808.321494:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff685f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074661248:0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.321499:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9247480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074661632:0@lo:400:kworker.0 00000100:00100000:1.0:1789762808.321510:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074661120 00000100:00100000:0.0:1789762808.321513:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9245180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074661760:0@lo:400:kworker.0 00000100:00100000:1.0:1789762808.321515:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074661120 00000100:00100000:2.0:1789762808.321525:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 243 00000100:00100000:3.0:1789762808.321530:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762808.321538:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074661376:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.321554:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1789762808.321558:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 244 00000100:00100000:3.0:1789762808.321564:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074660992:0@lo:400:kworker.0 00000100:00100000:0.0:1789762808.321571:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074661120:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762808.321575:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074661632 00000100:00100000:1.0:1789762808.321579:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074661632 00000100:00100000:2.0:1789762808.321581:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074661376:12345-0@lo:400:kworker.0 Request processed in 45us (434us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762808.321588:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 243 00000100:00100000:3.0:1789762808.321597:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1789762808.321611:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762808.321623:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074661120:12345-0@lo:400:kworker.0 Request processed in 55us (154us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762808.321623:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074660992 00000100:00100000:3.0:1789762808.321628:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9247b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074661504:0@lo:400:kworker.0 00000100:00100000:1.0:1789762808.321629:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074660992 00000100:00100000:0.0:1789762808.321635:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 244 00000100:00100000:2.0:1789762808.321646:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 374 00000100:00100000:2.0:1789762808.321655:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074661632:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.321661:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762808.321672:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074661504 00000100:00100000:0.0:1789762808.321677:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074661504 00000100:00100000:1.0:1789762808.321680:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 375 00000100:00100000:1.0:1789762808.321693:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074660992:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.321700:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074661120:0@lo:400:kworker.0 00000100:00100000:2.0:1789762808.321701:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074661632:12345-0@lo:400:kworker.0 Request processed in 46us (175us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762808.321715:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:3.0:1789762808.321720:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9247480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074661632:0@lo:400:kworker.0 00000100:00100000:2.0:1789762808.321722:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 374 00000100:00100000:0.0:1789762808.321728:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074661504:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762808.321741:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074660992:12345-0@lo:400:kworker.0 Request processed in 52us (144us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762808.321749:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 375 00000100:00080000:2.0:1789762808.321750:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762808.321756:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074660992:0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.321774:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9247b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074661504:0@lo:400:kworker.0 00000100:00100000:0.0:1789762808.321777:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074661504:12345-0@lo:400:kworker.0 Request processed in 53us (115us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762808.321807:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00000100:00100000:2.0:1789762808.321812:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762808.321831:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074661888 00000100:00100000:0.0:1789762808.321841:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074661888 00000100:00100000:2.0:1789762808.321847:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074660864:0@lo:400:kworker.0 00000100:00080000:3.0:1789762808.321865:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762808.321872:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9246300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074661376:0@lo:400:kworker.0 00000100:00100000:0.0:1789762808.321887:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 377 00000100:00100000:0.0:1789762808.321895:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074661888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762808.321952:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074661888:12345-0@lo:400:kworker.0 Request processed in 58us (146us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762808.321957:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 377 00000100:00100000:2.0:1789762808.321972:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9244a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074661888:0@lo:400:kworker.0 00010000:00010000:2.0:1789762808.448011:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762809.472096:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762810.368168:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074662016:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762810.368283:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762810.368348:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074662016 00000100:00100000:1.0:1789762810.368360:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074662016 00000100:00100000:1.0:1789762810.368437:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:1.0:1789762810.368453:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074662016:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762810.368501:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762810.368566:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074662016:12345-0@lo:41:osp-pre-1-0.0 Request processed in 118us (290us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762810.368592:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00100000:2.0:1789762810.368635:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074662016:0@lo:41:osp-pre-1-0.0 00010000:00010000:3.0:1789762810.496107:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762811.392105:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074662144:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789762811.392196:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762811.392250:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074662144 00000100:00100000:1.0:1789762811.392263:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074662144 00000100:00100000:1.0:1789762811.392337:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:1.0:1789762811.392354:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074662144:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789762811.392405:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789762811.392474:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074662144:12345-0@lo:41:osp-pre-0-1.0 Request processed in 127us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762811.392481:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00100000:0.0:1789762811.392505:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074662144:0@lo:41:osp-pre-0-1.0 00000001:00080000:3.0:1789762811.410705:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967674 is committed 00010000:00010000:1.0:1789762811.520115:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762812.359762:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762812.359802:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762812.359813:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067358208 00000100:00100000:0.0:1789762812.359826:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067358208 00000100:00100000:3.0:1789762812.359843:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762812.359861:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067358464 00000100:00100000:1.0:1789762812.359872:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067358464 00000100:00100000:0.0:1789762812.359892:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 337 00000100:00100000:3.0:1789762812.359898:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762812.359910:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067358208:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762812.359997:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067358336 00000100:00100000:3.0:1789762812.360005:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067358336 00000100:00100000:1.0:1789762812.360049:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 378 00000100:00100000:1.0:1789762812.360061:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067358464:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762812.360070:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 379 00000100:00100000:3.0:1789762812.360095:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067358336:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762812.360181:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067358464:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 109us (367us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762812.360184:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067358208:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 265us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762812.360188:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067358336:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 81us (331us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762812.360215:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 378 00000100:00100000:3.0:1789762812.360227:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 379 00000100:00100000:0.0:1789762812.360230:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 337 00000100:00080000:1.0:1789762812.360249:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762812.360255:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762812.360274:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067358592 00000100:00100000:1.0:1789762812.360278:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067358592 00000100:00100000:1.0:1789762812.360337:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 146 00000100:00100000:1.0:1789762812.360352:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067358592:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:0.0:1789762812.360375:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762812.360403:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067358592:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 56us (509us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762812.360411:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 146 00000100:00100000:0.0:1789762812.362053:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762812.362099:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762812.362101:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067358720 00000100:00100000:1.0:1789762812.362108:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067358720 00000100:00100000:3.0:1789762812.362153:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067358976 00000100:00100000:0.0:1789762812.362154:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762812.362162:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067358976 00000100:00100000:1.0:1789762812.362167:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 132 00000100:00100000:1.0:1789762812.362177:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2394:x1876702067358720:12345-192.168.203.37@tcp:9:kworker.0 00000100:00100000:0.0:1789762812.362193:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067358848 00000020:00000080:1.0:1789762812.362201:0:8053:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xdf5a7ac59f174d78 00000100:00100000:0.0:1789762812.362202:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067358848 00000100:00080000:1.0:1789762812.362214:0:8053:0:(import.c:65:import_set_state_nolock()) ffff9c37c8169000 : changing import state from FULL to CLOSED 00000100:00100000:3.0:1789762812.362222:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 380 00000100:00100000:3.0:1789762812.362237:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94e8850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067358976:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762812.362255:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 381 00000001:00000004:1.0:1789762812.362261:0:8053:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967363 00000100:00100000:0.0:1789762812.362269:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067358848:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762812.362282:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94e8850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067358976:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 49us (186us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762812.362288:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 380 00000100:00100000:0.0:1789762812.362314:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067358848:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 48us (163us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762812.362334:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 381 00000100:00080000:0.0:1789762812.362425:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762812.362433:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762812.362442:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1789762812.362490:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762812.362889:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+2:2394:x1876702067358720:12345-192.168.203.37@tcp:9:kworker.0 Request processed in 713us (837us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762812.362896:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 132 00010000:00010000:1.0:1789762812.544072:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762813.440120:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074662272:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762813.440136:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9247b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074662400:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762813.440252:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9247480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074662528:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762813.440260:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762813.440289:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074662656:0@lo:400:kworker.0 00000100:00100000:0.0:1789762813.440330:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074662272 00000100:00100000:0.0:1789762813.440337:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074662272 00000100:00100000:1.0:1789762813.440367:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762813.440411:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 158 00000100:00100000:0.0:1789762813.440426:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074662272:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762813.440435:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074662912:0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.440443:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074663168:0@lo:400:kworker.0 00002000:00000024:0.0:1789762813.440462:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00080000:3.0:1789762813.440516:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762813.440523:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762813.440532:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762813.440543:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074662272:12345-0@lo:13:osp-pre-0-0.0 Request processed in 112us (285us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762813.440556:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 158 00000100:00100000:1.0:1789762813.440568:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074663680:0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.440576:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074663424:0@lo:400:kworker.0 00000100:00100000:3.0:1789762813.440577:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074662912 00000100:00100000:3.0:1789762813.440588:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074662912 00000100:00100000:0.0:1789762813.440594:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762813.440600:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762813.440642:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074663040:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762813.440642:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074664192:0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.440648:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762813.440653:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 247 00000100:00100000:3.0:1789762813.440673:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074662912:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.440709:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074663936:0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.440717:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762813.440733:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762813.440733:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074662912:12345-0@lo:400:kworker.0 Request processed in 61us (221us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762813.440757:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 247 00000100:00100000:3.0:1789762813.440790:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074662528 00000100:00100000:0.0:1789762813.440792:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074663552:0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.440800:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762813.440801:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074662528 00000100:00100000:1.0:1789762813.440809:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074662656 00000100:00100000:1.0:1789762813.440815:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074662656 00000100:00100000:3.0:1789762813.440865:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074663040 00000100:00100000:0.0:1789762813.440869:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762813.440871:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074663040 00000100:00100000:1.0:1789762813.440874:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 338 00000100:00100000:2.0:1789762813.440878:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967674 gen 2 00000004:00080000:2.0:1789762813.440887:0:3427:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37d8c9e680, transno 4294967674 00000100:00100000:1.0:1789762813.440889:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074662656:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762813.440910:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074664064:0@lo:400:kworker.0 00000100:00100000:3.0:1789762813.440911:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 159 00000100:00100000:3.0:1789762813.440956:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074662528:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762813.440965:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074662272:0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762813.440968:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762813.440988:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074662656:12345-0@lo:400:kworker.0 Request processed in 105us (625us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762813.440997:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 338 00000100:00100000:2.0:1789762813.441013:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074662912:0@lo:400:kworker.0 00000100:00100000:0.0:1789762813.441014:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762813.441043:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074662528:12345-0@lo:13:osp-pre-0-1.0 Request processed in 87us (445us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762813.441044:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [2->3] 00000100:00100000:1.0:1789762813.441054:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074663168 00000100:00100000:1.0:1789762813.441060:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074663168 00000100:00100000:2.0:1789762813.441064:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074662784:0@lo:400:kworker.0 00000100:00100000:3.0:1789762813.441065:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 159 00000100:00100000:0.0:1789762813.441083:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9247480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074662528:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762813.441109:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074663680 00000100:00100000:3.0:1789762813.441110:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:1.0:1789762813.441118:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074663680 00000100:00100000:2.0:1789762813.441125:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762813.441135:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9247800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074663040:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762813.441161:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074663424 00000100:00100000:1.0:1789762813.441163:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074663552 00002000:00000024:3.0:1789762813.441168:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762813.441169:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074663552 00000100:00100000:2.0:1789762813.441171:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074663296:0@lo:400:kworker.0 00000100:00100000:0.0:1789762813.441175:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074663424 00000100:00100000:1.0:1789762813.441198:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074662784 00000100:00100000:1.0:1789762813.441205:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074662784 00000100:00100000:2.0:1789762813.441219:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762813.441229:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:0.0:1789762813.441242:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074663168:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.441247:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000100:00100000:2.0:1789762813.441264:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074663808:0@lo:400:kworker.0 00000100:00100000:3.0:1789762813.441270:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9247800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074663040:12345-0@lo:13:osp-pre-1-0.0 Request processed in 133us (541us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762813.441275:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074663680:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762813.441292:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:0.0:1789762813.441293:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074663168:12345-0@lo:400:kworker.0 Request processed in 50us (776us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762813.441314:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:2.0:1789762813.441323:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1789762813.441335:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762813.441347:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762813.441353:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074663808 00000100:00080000:3.0:1789762813.441354:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762813.441359:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074663808 00000100:00080000:3.0:1789762813.441361:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762813.441363:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074664320:0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.441365:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074663680:12345-0@lo:400:kworker.0 Request processed in 104us (763us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762813.441381:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00000100:00100000:3.0:1789762813.441389:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:3.0:1789762813.441402:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074663552:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.441408:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:0.0:1789762813.441415:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:1.0:1789762813.441423:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074663424:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.441431:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762813.441433:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074662784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762813.441462:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074663552:12345-0@lo:400:kworker.0 Request processed in 63us (598us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762813.441474:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:1.0:1789762813.441507:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074663424:12345-0@lo:400:kworker.0 Request processed in 88us (868us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762813.441517:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:0.0:1789762813.441522:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074662784:12345-0@lo:400:kworker.0 Request processed in 95us (409us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762813.441528:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074663808:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.441532:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00100000:0.0:1789762813.441546:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00100000:2.0:1789762813.441565:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074664192 00000100:00100000:2.0:1789762813.441575:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074664192 00000100:00100000:1.0:1789762813.441576:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074663296 00000100:00100000:3.0:1789762813.441586:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074663808:12345-0@lo:400:kworker.0 Request processed in 55us (276us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762813.441588:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762813.441590:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074663296 00000100:00100000:3.0:1789762813.441598:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00000100:00100000:0.0:1789762813.441605:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762813.441659:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 248 00000100:00100000:2.0:1789762813.441661:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074663936 00000100:00100000:2.0:1789762813.441667:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074663936 00000100:00100000:0.0:1789762813.441671:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074662400 00000100:00100000:1.0:1789762813.441672:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074663296:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762813.441680:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074662400 00000100:00100000:2.0:1789762813.441701:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074664064 00000100:00100000:2.0:1789762813.441706:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074664064 00000100:00100000:1.0:1789762813.441722:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074663296:12345-0@lo:400:kworker.0 Request processed in 55us (503us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762813.441734:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 248 00000100:00100000:0.0:1789762813.441760:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:2.0:1789762813.441763:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074664320 00000100:00100000:0.0:1789762813.441775:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9247480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074662400:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762813.441778:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074664320 00000100:00100000:1.0:1789762813.441784:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 147 00002000:00000024:0.0:1789762813.441796:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762813.441801:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074664192:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.441831:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:2.0:1789762813.441844:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074663936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762813.441867:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9247480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074662400:12345-0@lo:13:osp-pre-1-1.0 Request processed in 92us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762813.441867:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074664192:12345-0@lo:400:kworker.0 Request processed in 71us (1152us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762813.441879:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000100:00100000:1.0:1789762813.441889:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 147 00000100:00100000:2.0:1789762813.441912:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074663936:12345-0@lo:400:kworker.0 Request processed in 72us (1123us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762813.441952:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000100:00080000:0.0:1789762813.441953:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762813.441963:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:1.0:1789762813.441982:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074664064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.441997:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:0.0:1789762813.442005:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9247b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074662400:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762813.442010:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074664320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.442039:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074664064:12345-0@lo:400:kworker.0 Request processed in 65us (1030us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762813.442052:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000100:00100000:0.0:1789762813.442064:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074663040:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762813.442073:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074664320:12345-0@lo:400:kworker.0 Request processed in 66us (654us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762813.442083:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:0.0:1789762813.442089:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074663552:0@lo:400:kworker.0 00000100:00100000:0.0:1789762813.442122:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074664064:0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.442129:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074662656:0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.442167:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074663168:0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.442179:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074662784:0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.442202:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074663680:0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.442207:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074663296:0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.442242:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074663808:0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.442253:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074664192:0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.442257:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074664320:0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.442275:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074663424:0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.442287:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074663936:0@lo:400:kworker.0 00010000:00010000:3.0:1789762813.568426:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762814.592080:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762815.488063:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074664448:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762815.488131:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762815.488179:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074664448 00000100:00100000:0.0:1789762815.488187:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074664448 00000100:00100000:0.0:1789762815.488238:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 249 00000100:00100000:0.0:1789762815.488248:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074664448:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762815.488281:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762815.488332:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074664448:12345-0@lo:41:osp-pre-1-0.0 Request processed in 87us (202us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762815.488340:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 249 00000100:00100000:2.0:1789762815.488387:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074664448:0@lo:41:osp-pre-1-0.0 00000001:00100000:0.0:1789762815.496133:0:9000:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000020:00000080:0.0:1789762815.496139:0:9000:0:(genops.c:885:class_export_put()) final put ffff9c37c816a800/fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000020:00000080:3.0:1789762815.496168:0:36:0:(genops.c:844:class_export_destroy()) destroying export ffff9c37c816a800/fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 for lustre-OST0001 00000020:00000080:3.0:1789762815.496217:0:36:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9c37c8169000 for lustre-OST0001 00010000:00010000:0.0:1789762815.616040:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762816.512132:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074664576:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762816.512202:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762816.512289:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074664576 00000100:00100000:2.0:1789762816.512298:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074664576 00000100:00100000:2.0:1789762816.512357:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 250 00000100:00100000:2.0:1789762816.512385:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074664576:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762816.512418:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762816.512469:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074664576:12345-0@lo:41:osp-pre-0-1.0 Request processed in 88us (267us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762816.512475:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 250 00000100:00100000:1.0:1789762816.512528:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074664576:0@lo:41:osp-pre-0-1.0 00010000:00010000:1.0:1789762816.640053:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762817.480000:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762817.480031:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067359104 00000100:00100000:0.0:1789762817.480032:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762817.480054:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762817.480054:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067359104 00000100:00100000:2.0:1789762817.480081:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067359232 00000100:00100000:0.0:1789762817.480086:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762817.480088:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067359232 00000100:00100000:3.0:1789762817.480111:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 339 00000100:00100000:0.0:1789762817.480119:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762817.480121:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067359488 00000100:00100000:3.0:1789762817.480127:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067359104:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762817.480129:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067359488 00000100:00100000:2.0:1789762817.480145:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067359360 00000100:00100000:0.0:1789762817.480151:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762817.480152:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067359360 00000100:00080000:0.0:1789762817.480176:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762817.480180:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 151 00000100:00080000:0.0:1789762817.480183:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762817.480187:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762817.480193:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067359488:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762817.480193:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067359744 00000100:00100000:3.0:1789762817.480195:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067359104:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 74us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762817.480200:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067359744 00000100:00100000:0.0:1789762817.480201:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067359616 00000100:00100000:0.0:1789762817.480208:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067359616 00000100:00100000:3.0:1789762817.480209:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 339 00000100:00100000:1.0:1789762817.480239:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067359488:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 50us (183us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762817.480246:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 151 00000100:00100000:2.0:1789762817.480251:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 388 00000100:00100000:0.0:1789762817.480261:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 389 00000100:00100000:2.0:1789762817.480265:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9247b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067359232:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762817.480271:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 390 00000100:00100000:1.0:1789762817.480280:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 391 00000100:00100000:0.0:1789762817.480281:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9247100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067359360:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762817.480282:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067359744:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762817.480302:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067359616:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762817.480314:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9247b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067359232:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 55us (281us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762817.480321:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067359744:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 42us (204us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762817.480326:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 388 00000100:00100000:3.0:1789762817.480337:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 390 00000100:00100000:0.0:1789762817.480338:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9247100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067359360:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 63us (250us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762817.480339:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067359616:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 40us (187us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762817.480353:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 391 00000100:00100000:0.0:1789762817.480365:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 389 00000100:00080000:1.0:1789762817.480393:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762817.480398:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:3.0:1789762817.664064:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762818.560080:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074665088:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762818.560090:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074664832:0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.560093:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074665216:0@lo:400:kworker.0 00000100:00100000:2.0:1789762818.560165:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762818.560173:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762818.560173:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762818.560251:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074664704:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762818.560260:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074665728:0@lo:400:kworker.0 00000100:00100000:0.0:1789762818.560263:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074665600:0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.560289:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074665088 00000100:00100000:2.0:1789762818.560290:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762818.560300:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074665088 00000100:00100000:1.0:1789762818.560313:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762818.560323:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762818.560328:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074665344:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762818.560356:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074666240:0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.560360:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074664704 00000100:00100000:0.0:1789762818.560361:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074666112:0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.560366:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074664704 00000100:00100000:2.0:1789762818.560368:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762818.560394:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074665344 00000100:00100000:3.0:1789762818.560399:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074665344 00000100:00100000:1.0:1789762818.560415:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762818.560419:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074665856:0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.560428:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 162 00000100:00100000:0.0:1789762818.560438:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:3.0:1789762818.560442:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074665088:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762818.560449:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfefb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074666752:0@lo:400:kworker.0 00000100:00100000:0.0:1789762818.560457:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074664704:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762818.560457:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:3.0:1789762818.560478:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762818.560485:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074666368:0@lo:400:kworker.0 00002000:00000024:0.0:1789762818.560487:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762818.560493:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762818.560532:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074665088:12345-0@lo:13:osp-pre-1-1.0 Request processed in 93us (369us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762818.560542:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 162 00000100:00100000:1.0:1789762818.560547:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074664832 00000100:00100000:2.0:1789762818.560550:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762818.560551:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074664704:12345-0@lo:13:osp-pre-0-0.0 Request processed in 99us (260us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762818.560553:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074664832 00000100:00100000:3.0:1789762818.560562:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:0.0:1789762818.560570:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:3.0:1789762818.560576:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074665344:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762818.560598:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074664704:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762818.560599:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 340 00002000:00000024:3.0:1789762818.560603:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762818.560611:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff685c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074664960:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762818.560613:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074664832:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762818.560643:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074665728 00000100:00100000:2.0:1789762818.560648:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074665728 00000100:00100000:3.0:1789762818.560656:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074665344:12345-0@lo:13:osp-pre-0-1.0 Request processed in 84us (291us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762818.560667:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762818.560671:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00100000:1.0:1789762818.560674:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074664832:12345-0@lo:400:kworker.0 Request processed in 62us (514us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762818.560681:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 340 00000100:00100000:2.0:1789762818.560693:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 251 00000100:00100000:3.0:1789762818.560699:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074664960 00000100:00100000:0.0:1789762818.560718:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074665472:0@lo:400:kworker.0 00000100:00100000:2.0:1789762818.560720:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074665728:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.560720:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074664960 00000100:00100000:1.0:1789762818.560733:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074665216 00000100:00100000:1.0:1789762818.560738:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074665216 00000100:00100000:3.0:1789762818.560758:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:0.0:1789762818.560770:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762818.560772:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074665600 00000100:00100000:3.0:1789762818.560772:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074664960:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762818.560777:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074665600 00000100:00100000:2.0:1789762818.560780:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074665728:12345-0@lo:400:kworker.0 Request processed in 78us (467us total) trans 0 rc 0/0 00002000:00000024:3.0:1789762818.560786:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762818.560791:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 251 00000100:00100000:0.0:1789762818.560803:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074665984:0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.560806:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074666240 00000100:00100000:2.0:1789762818.560811:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074665472 00000100:00100000:2.0:1789762818.560816:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074665472 00000100:00100000:1.0:1789762818.560819:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074666240 00000100:00100000:3.0:1789762818.560831:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074664960:12345-0@lo:13:osp-pre-1-0.0 Request processed in 65us (169us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762818.560840:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:0.0:1789762818.560848:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762818.560854:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:1.0:1789762818.560855:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074665856 00000100:00100000:1.0:1789762818.560860:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074665856 00000100:00100000:2.0:1789762818.560867:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074665472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762818.560869:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074666496:0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.560880:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074665984 00000100:00100000:3.0:1789762818.560881:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074666752 00000100:00100000:1.0:1789762818.560885:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074665984 00000100:00100000:3.0:1789762818.560885:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074666752 00000100:00100000:1.0:1789762818.560909:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:3.0:1789762818.560939:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074666368 00000100:00100000:2.0:1789762818.560940:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074665472:12345-0@lo:400:kworker.0 Request processed in 76us (174us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762818.560943:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762818.560944:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074665216:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.560944:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074666368 00000100:00100000:2.0:1789762818.560953:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00100000:3.0:1789762818.560977:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074666496 00000100:00100000:3.0:1789762818.560982:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074666496 00000100:00100000:2.0:1789762818.560984:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1789762818.560995:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074666752:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.561004:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074665216:12345-0@lo:400:kworker.0 Request processed in 86us (840us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762818.561007:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff685c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074664960:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762818.561010:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:1.0:1789762818.561016:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00100000:3.0:1789762818.561024:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074666368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762818.561035:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074666752:12345-0@lo:400:kworker.0 Request processed in 42us (545us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762818.561037:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074665472:0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.561042:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:2.0:1789762818.561051:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00000100:00100000:1.0:1789762818.561057:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff684e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074665600:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762818.561076:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:2.0:1789762818.561086:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff684000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074666496:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.561093:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074666368:12345-0@lo:400:kworker.0 Request processed in 72us (546us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762818.561097:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff684e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074665600:12345-0@lo:400:kworker.0 Request processed in 45us (778us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762818.561104:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:3.0:1789762818.561104:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00100000:0.0:1789762818.561118:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1789762818.561127:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:0.0:1789762818.561134:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074666240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762818.561136:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff684000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074666496:12345-0@lo:400:kworker.0 Request processed in 50us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762818.561137:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:1.0:1789762818.561140:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074665856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.561149:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074665984:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762818.561164:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:0.0:1789762818.561204:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074666240:12345-0@lo:400:kworker.0 Request processed in 75us (788us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762818.561209:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074665856:12345-0@lo:400:kworker.0 Request processed in 70us (751us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762818.561209:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff687100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074665984:12345-0@lo:400:kworker.0 Request processed in 59us (362us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762818.561216:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074665984:0@lo:400:kworker.0 00000100:00100000:0.0:1789762818.561219:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1789762818.561228:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:3.0:1789762818.561234:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:2.0:1789762818.561245:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074666496:0@lo:400:kworker.0 00000100:00080000:1.0:1789762818.561272:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762818.561277:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789762818.561280:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789762818.561282:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762818.561289:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762818.561292:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074665856:0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.561310:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074665728:0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.561310:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074666368:0@lo:400:kworker.0 00000100:00100000:0.0:1789762818.561323:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074666624:0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.561325:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074666240:0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.561325:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074664832:0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.561341:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfefb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074666752:0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.561346:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074665344:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762818.561361:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762818.561364:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074665216:0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.561407:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074666112 00000100:00100000:0.0:1789762818.561410:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074665088:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762818.561412:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074666112 00000100:00100000:2.0:1789762818.561417:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074666624 00000100:00100000:2.0:1789762818.561422:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074666624 00000100:00100000:0.0:1789762818.561442:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074665600:0@lo:400:kworker.0 00000100:00100000:2.0:1789762818.561451:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:2.0:1789762818.561461:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff684700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074666624:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789762818.561467:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762818.561469:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:1.0:1789762818.561479:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074666112:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762818.561507:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff684700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074666624:12345-0@lo:400:kworker.0 Request processed in 47us (147us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762818.561513:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:0.0:1789762818.561523:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074666624:0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.561536:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff685c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074666112:12345-0@lo:400:kworker.0 Request processed in 58us (248us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762818.561545:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:0.0:1789762818.561548:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074666112:0@lo:400:kworker.0 00010000:00010000:1.0:1789762818.688067:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762819.712087:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762820.608080:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074666880:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762820.608158:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762820.608197:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074666880 00000100:00100000:3.0:1789762820.608206:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074666880 00000100:00100000:3.0:1789762820.608264:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 253 00000100:00100000:3.0:1789762820.608278:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074666880:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789762820.608320:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:3.0:1789762820.608380:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074666880:12345-0@lo:41:osp-pre-1-0.0 Request processed in 106us (223us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762820.608390:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 253 00000100:00100000:1.0:1789762820.608415:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074666880:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1789762820.736040:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762821.632156:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074667008:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762821.632263:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762821.632332:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074667008 00000100:00100000:2.0:1789762821.632341:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074667008 00000100:00100000:2.0:1789762821.632408:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 254 00000100:00100000:2.0:1789762821.632423:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074667008:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762821.632465:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762821.632551:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074667008:12345-0@lo:41:osp-pre-0-1.0 Request processed in 135us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762821.632560:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 254 00000100:00100000:3.0:1789762821.632614:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074667008:0@lo:41:osp-pre-0-1.0 00010000:00010000:3.0:1789762821.760109:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762822.599964:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762822.600013:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067360000 00000100:00100000:0.0:1789762822.600019:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762822.600025:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067360000 00000100:00100000:0.0:1789762822.600046:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762822.600068:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067359872 00000100:00100000:0.0:1789762822.600075:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762822.600075:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067359872 00000100:00100000:0.0:1789762822.600104:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762822.600105:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067360128 00000100:00080000:2.0:1789762822.600109:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762822.600111:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067360128 00000100:00100000:1.0:1789762822.600124:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 341 00000100:00100000:2.0:1789762822.600131:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067360256 00000100:00100000:1.0:1789762822.600137:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067359872:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762822.600138:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067360256 00000100:00100000:3.0:1789762822.600146:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067360384 00000100:00100000:0.0:1789762822.600148:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762822.600155:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067360384 00000100:00100000:0.0:1789762822.600181:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067360512 00000100:00100000:0.0:1789762822.600187:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067360512 00000100:00100000:2.0:1789762822.600192:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 156 00000100:00100000:3.0:1789762822.600200:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 398 00000100:00100000:2.0:1789762822.600204:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff687480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067360256:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762822.600214:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067359872:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 80us (197us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762822.600224:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 341 00000100:00100000:3.0:1789762822.600231:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff687b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067360000:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762822.600237:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 399 00000100:00100000:2.0:1789762822.600245:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff687480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067360256:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 44us (169us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762822.600252:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067360128:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762822.600264:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 400 00000100:00100000:2.0:1789762822.600270:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 156 00000100:00100000:3.0:1789762822.600274:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff687b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067360000:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 47us (318us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762822.600276:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067360384:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762822.600290:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 398 00000100:00100000:0.0:1789762822.600292:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff686300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067360128:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 44us (244us total) trans 0 rc 0/0 00000100:00080000:2.0:1789762822.600297:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762822.600303:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762822.600309:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762822.600310:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 399 00000100:00100000:1.0:1789762822.600316:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff685f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067360384:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 43us (212us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762822.600322:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 401 00000100:00100000:3.0:1789762822.600332:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067360512:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762822.600335:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 400 00000100:00100000:3.0:1789762822.600363:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067360512:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 33us (223us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762822.600369:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 401 00010000:00010000:1.0:1789762822.784061:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762823.680076:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074667264:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762823.680083:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074667392:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762823.680103:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074667136:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762823.680141:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074667520:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762823.680155:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762823.680187:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762823.680216:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762823.680240:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074667776:0@lo:400:kworker.0 00000100:00100000:2.0:1789762823.680242:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074667648:0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.680254:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074668032:0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.680290:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762823.680309:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762823.680316:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074667264 00000100:00100000:1.0:1789762823.680324:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074667264 00000100:00100000:3.0:1789762823.680326:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074667136 00000100:00100000:0.0:1789762823.680331:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074668288:0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.680332:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074667136 00000100:00100000:2.0:1789762823.680351:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074668160:0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.680364:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762823.680386:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762823.680389:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074667520 00000100:00100000:0.0:1789762823.680397:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff880380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074668800:0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.680398:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:1.0:1789762823.680404:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074667520 00000100:00100000:2.0:1789762823.680410:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff881180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074668672:0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.680415:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074667264:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762823.680434:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762823.680438:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 167 00000100:00100000:2.0:1789762823.680448:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:3.0:1789762823.680452:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762823.680453:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074667136:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762823.680460:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1789762823.680470:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074667904:0@lo:400:kworker.0 00002000:00000024:1.0:1789762823.680479:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762823.680481:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789762823.680490:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff880a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074668544:0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.680504:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762823.680509:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074667264:12345-0@lo:13:osp-pre-0-1.0 Request processed in 99us (356us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762823.680516:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:1.0:1789762823.680538:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074667136:12345-0@lo:13:osp-pre-1-0.0 Request processed in 90us (350us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762823.680538:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762823.680542:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074668416:0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.680543:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:1.0:1789762823.680557:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 167 00000100:00100000:3.0:1789762823.680558:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074667520:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762823.680570:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241762304 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762823.680579:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762823.680586:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074668800 00000100:00100000:1.0:1789762823.680591:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074668800 00000100:00100000:2.0:1789762823.680594:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074667264:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762823.680612:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074667520:12345-0@lo:13:osp-pre-0-0.0 Request processed in 57us (398us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762823.680630:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074667136:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762823.680635:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:3.0:1789762823.680636:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00100000:2.0:1789762823.680638:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074668160 00000100:00100000:2.0:1789762823.680646:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074668160 00000100:00100000:1.0:1789762823.680651:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074668800:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.680691:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074667904 00000100:00100000:0.0:1789762823.680696:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074667776 00000100:00100000:3.0:1789762823.680696:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074667904 00000100:00100000:0.0:1789762823.680702:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074667776 00000100:00100000:2.0:1789762823.680706:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 255 00000100:00100000:1.0:1789762823.680714:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074668800:12345-0@lo:400:kworker.0 Request processed in 66us (280us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762823.680727:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:2.0:1789762823.680727:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074668160:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.680744:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074668288 00000100:00100000:0.0:1789762823.680747:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074668288 00000100:00100000:3.0:1789762823.680749:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 256 00000100:00100000:1.0:1789762823.680756:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074668672 00000100:00100000:1.0:1789762823.680762:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074668672 00000100:00100000:3.0:1789762823.680766:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074667904:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.680772:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074668544 00000100:00100000:0.0:1789762823.680776:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074668544 00000100:00100000:2.0:1789762823.680781:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074668160:12345-0@lo:400:kworker.0 Request processed in 66us (395us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762823.680788:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 255 00000100:00100000:1.0:1789762823.680791:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074668416 00000100:00100000:1.0:1789762823.680796:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074668416 00000100:00100000:0.0:1789762823.680809:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:3.0:1789762823.680817:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074667904:12345-0@lo:400:kworker.0 Request processed in 57us (312us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762823.680822:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074667776:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762823.680832:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1789762823.680832:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 256 00000100:00100000:1.0:1789762823.680842:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:2.0:1789762823.680847:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074668288:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762823.680857:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074668672:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.680863:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074667776:12345-0@lo:400:kworker.0 Request processed in 46us (573us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762823.680870:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762823.680883:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:0.0:1789762823.680907:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074668032 00000100:00100000:3.0:1789762823.680908:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074667520:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762823.680912:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074668672:12345-0@lo:400:kworker.0 Request processed in 59us (463us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762823.680915:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074668032 00000100:00100000:2.0:1789762823.680943:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074668288:12345-0@lo:400:kworker.0 Request processed in 74us (555us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762823.680952:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00100000:2.0:1789762823.680960:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:1.0:1789762823.680978:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:3.0:1789762823.680984:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:1.0:1789762823.680989:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074668544:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.680994:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:3.0:1789762823.680998:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074668416:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.681009:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074668032:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762823.681042:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074668160:0@lo:400:kworker.0 00000100:00100000:1.0:1789762823.681043:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074668544:12345-0@lo:400:kworker.0 Request processed in 57us (506us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762823.681053:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074668416:12345-0@lo:400:kworker.0 Request processed in 59us (473us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762823.681059:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:2.0:1789762823.681065:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff881180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074668672:0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.681067:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:0.0:1789762823.681074:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074668032:12345-0@lo:400:kworker.0 Request processed in 65us (201us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762823.681084:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00100000:1.0:1789762823.681094:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1789762823.681096:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762823.681101:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff880a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074668544:0@lo:400:kworker.0 00000100:00080000:3.0:1789762823.681101:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789762823.681108:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789762823.681114:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762823.681131:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074667648 00000100:00100000:3.0:1789762823.681134:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074667648 00000100:00100000:1.0:1789762823.681156:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074667392 00000100:00100000:0.0:1789762823.681160:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074667904:0@lo:400:kworker.0 00000100:00100000:1.0:1789762823.681163:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074667392 00000100:00100000:2.0:1789762823.681166:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074668032:0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.681170:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 342 00000100:00100000:3.0:1789762823.681185:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1579c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074667648:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.681187:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074668416:0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.681208:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074668288:0@lo:400:kworker.0 00000100:00100000:1.0:1789762823.681209:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:1.0:1789762823.681225:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074667392:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762823.681227:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff880380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074668800:0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.681239:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1579c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074667648:12345-0@lo:400:kworker.0 Request processed in 57us (930us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762823.681246:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 342 00000100:00100000:0.0:1789762823.681248:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074667776:0@lo:400:kworker.0 00002000:00000024:1.0:1789762823.681248:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762823.681262:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074667648:0@lo:400:kworker.0 00000100:00100000:1.0:1789762823.681306:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074667392:12345-0@lo:13:osp-pre-1-1.0 Request processed in 85us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762823.681313:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000100:00100000:2.0:1789762823.681328:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074667392:0@lo:13:osp-pre-1-1.0 00010000:00010000:0.0:1789762823.808047:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762824.832084:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762825.074075:0:15781:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.37@tcp 00000100:00100000:3.0:1789762825.074095:0:15781:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:3.0:1789762825.074197:0:15781:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762825.074207:0:15781:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.37@tcp 00000100:00100000:3.0:1789762825.074221:0:15781:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:3.0:1789762825.074227:0:15781:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c36c2126b00 rate 20 gen 0 00000100:00100000:3.0:1789762825.074231:0:15781:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762825.074234:0:15781:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.37@tcp 00000100:00100000:3.0:1789762825.074276:0:15781:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:3.0:1789762825.074280:0:15781:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c36c2126500 rate 20 gen 0 00000100:00100000:3.0:1789762825.074386:0:15781:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.37@tcp 00000100:00100000:3.0:1789762825.074521:0:15781:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:3.0:1789762825.074527:0:15781:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762825.074531:0:15781:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.37@tcp 00000100:00100000:3.0:1789762825.074538:0:15781:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:3.0:1789762825.074541:0:15781:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c36c2126400 rate 10 gen 0 00000100:00100000:3.0:1789762825.074543:0:15781:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762825.074546:0:15781:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.37@tcp 00000100:00100000:3.0:1789762825.074554:0:15781:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:3.0:1789762825.074556:0:15781:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c36c2126a00 rate 10 gen 0 00000100:00100000:0.0:1789762825.085877:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762825.085947:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067360640 00000100:00100000:1.0:1789762825.085957:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067360640 00000100:00080000:3.0:1789762825.085993:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762825.086016:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 205 00000100:00100000:1.0:1789762825.086027:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:13911:x1876702067360640:12345-192.168.203.37@tcp:101:mkdir.0 00010000:00010000:1.0:1789762825.086043:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762825.087644:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755d6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6a3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.087720:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1755dd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.087741:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1755dd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.087750:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1755dd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762825.087781:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762825.087784:0:6420:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:1.0:1789762825.087799:0:6420:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1755dd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.087821:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1755d6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6a3 expref: 7 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.087833:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1755dd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6a3 expref: 6 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.087840:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762825.087843:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762825.087845:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d9230a00, rc 0) 00000100:00100000:1.0:1789762825.087899:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13911:x1876702067360640:12345-192.168.203.37@tcp:101:mkdir.0 Request processed in 1875us (2025us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762825.087905:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 205 00000100:00100000:2.0:1789762825.089044:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.089070:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067360896 00000100:00100000:3.0:1789762825.089078:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067360896 00000100:00080000:2.0:1789762825.089105:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762825.089132:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 206 00000100:00100000:3.0:1789762825.089143:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:13911:x1876702067360896:12345-192.168.203.37@tcp:36:mkdir.0 00000100:00100000:3.0:1789762825.089217:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074668928:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.089261:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762825.089298:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074668928 00000100:00100000:2.0:1789762825.089305:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074668928 00000100:00100000:3.0:1789762825.089306:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.089339:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 257 00000100:00100000:2.0:1789762825.089347:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfeca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:6419:x1876702074668928:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762825.089403:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfeca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:6419:x1876702074668928:12345-0@lo:1000:mdt00_001.0 Request processed in 57us (140us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.089409:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 257 00000100:00100000:3.0:1789762825.089425:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074668928:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.089478:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074669056:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.089504:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762825.089521:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.089523:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074669056 00000100:00100000:2.0:1789762825.089527:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074669056 00000100:00100000:2.0:1789762825.089550:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 258 00000100:00100000:2.0:1789762825.089555:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6419:x1876702074669056:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762825.089758:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6419:x1876702074669056:12345-0@lo:1000:mdt00_001.0 Request processed in 203us (253us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789762825.089770:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 258 00000100:00100000:3.0:1789762825.089783:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074669056:0@lo:1000:mdt00_001.0 00010000:00010000:3.0:1789762825.089816:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.089823:0:6419:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.089830:0:6419:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762825.089838:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074669184:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1789762825.089856:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762825.089873:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.089877:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074669184 00000100:00100000:2.0:1789762825.089882:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074669184 00000100:00100000:2.0:1789762825.089903:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 259 00000100:00100000:2.0:1789762825.089909:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfeed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6419:x1876702074669184:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1789762825.089938:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762825.089952:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f1755eb lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f1755e4 expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.089975:0:6424:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1755dd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6a3 expref: 5 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.089988:0:6424:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f1755eb lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f1755e4 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.090010:0:6424:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1755dd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6a3 expref: 5 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.090019:0:6424:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1755dd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6a3 expref: 5 pid: 6420 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762825.090027:0:6424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9ed80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074669312:192.168.203.37@tcp:104: 00000100:00100000:2.0:1789762825.090069:0:6424:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ee88d6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.090753:0:6424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9ed80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074669312:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762825.090771:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f1755eb lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1755e4 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.090787:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762825.090788:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762825.090792:0:6424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f1755eb lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1755e4 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.090797:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d10afa00, rc 0) 00000100:00100000:2.0:1789762825.090826:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfeed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6419:x1876702074669184:12345-0@lo:101:mdt00_001.0 Request processed in 917us (969us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.090831:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 259 00000100:00100000:3.0:1789762825.090842:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074669184:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1789762825.090901:0:6419:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.090910:0:6419:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762825.090990:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762825.091024:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067361024 00000100:00100000:1.0:1789762825.091030:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067361024 00010000:00010000:1.0:1789762825.091052:0:6406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1755dd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6a3 expref: 6 pid: 6420 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762825.091080:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 69 00000100:00100000:1.0:1789762825.091097:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067361024:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762825.091107:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762825.091112:0:6406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1755dd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6a3 expref: 6 pid: 6420 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.091125:0:6406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1755dd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6a3 expref: 6 pid: 6420 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.091138:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1755dd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6a3 expref: 6 pid: 6420 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.091147:0:6406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762825.091152:0:6406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f1755eb lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1755e4 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.091160:0:6406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f1755eb lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f1755e4 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.091165:0:6406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f1755eb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f1755e4 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.091185:0:6406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f1755eb lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1755e4 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762825.091193:0:6406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff882d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6406:1876702074669440:0@lo:105: 00000100:00100000:1.0:1789762825.091230:0:6406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762825.091261:0:6403:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074669440 00000100:00100000:2.0:1789762825.091267:0:6403:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074669440 00000100:00100000:1.0:1789762825.091270:0:6406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7b7e300 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.091309:0:6403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 45 00000100:00100000:2.0:1789762825.091317:0:6403:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6406:x1876702074669440:12345-0@lo:105: 00010000:00010000:2.0:1789762825.091324:0:6403:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.091332:0:6403:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.091338:0:6403:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.091343:0:6403:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.091349:0:6403:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.091361:0:6403:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.091375:0:6403:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c37f7d93600) 00010000:00010000:3.0:1789762825.091385:0:6419:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.091394:0:6419:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762825.091410:0:6403:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6406:x1876702074669440:12345-0@lo:105: Request processed in 93us (180us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.091417:0:6403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 45 00000004:00080000:3.0:1789762825.091421:0:6419:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1789762825.091438:0:6406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff882d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6406:1876702074669440:0@lo:105: 00010000:00010000:3.0:1789762825.091443:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.091449:0:6419:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.091450:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762825.091458:0:6419:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762825.091469:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074669568:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1789762825.091485:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067361024:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 390us (495us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762825.091497:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 69 00000100:00100000:3.0:1789762825.091498:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762825.091525:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f600 going to sleep for 11 seconds 00000100:00100000:1.0:1789762825.091537:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074669568 00000100:00100000:1.0:1789762825.091541:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074669568 00000100:00100000:1.0:1789762825.091564:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 260 00000100:00100000:1.0:1789762825.091572:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6419:x1876702074669568:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1789762825.091579:0:7227:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762825.091590:0:7227:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f1755f9 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f1755f2 expref: -99 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.091603:0:7227:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f1755f9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f1755f2 expref: 8 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.091612:0:7227:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f1755f9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1755f2 expref: 8 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.091617:0:7227:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743500) 00010000:00010000:1.0:1789762825.091619:0:7227:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762825.091621:0:7227:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d9231000, rc 0) 00000100:00100000:1.0:1789762825.091653:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074669568:12345-0@lo:101:mdt00_001.0 Request processed in 82us (158us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762825.091673:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 260 00000100:00100000:3.0:1789762825.091685:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074669568:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1789762825.091698:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1755f9 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.091706:0:6419:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1755f9 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762825.091715:0:6419:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1789762825.091732:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074669696:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.091754:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762825.091773:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.091785:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074669696 00000100:00100000:2.0:1789762825.091789:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074669696 00000100:00100000:2.0:1789762825.091814:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 261 00000100:00100000:2.0:1789762825.091820:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfefb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074669696:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762825.091868:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfefb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074669696:12345-0@lo:1000:mdt00_001.0 Request processed in 48us (113us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.091875:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 261 00000100:00100000:3.0:1789762825.091882:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074669696:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.091937:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074669824:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.091966:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762825.091987:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.091994:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074669824 00000100:00100000:2.0:1789762825.091998:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074669824 00000100:00100000:2.0:1789762825.092023:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 262 00000100:00100000:2.0:1789762825.092030:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074669824:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762825.092152:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074669824:12345-0@lo:1000:mdt00_001.0 Request processed in 123us (184us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789762825.092158:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 262 00000100:00100000:3.0:1789762825.092181:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074669824:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.092239:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074669952:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.092264:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762825.092281:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.092287:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074669952 00000100:00100000:2.0:1789762825.092292:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074669952 00000100:00100000:2.0:1789762825.092321:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 263 00000100:00100000:2.0:1789762825.092327:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074669952:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762825.092436:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074669952:12345-0@lo:1000:mdt00_001.0 Request processed in 109us (169us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789762825.092448:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074669952:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762825.092449:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 263 00000100:00100000:3.0:1789762825.092497:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074670080:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.092521:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762825.092536:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f900 going to sleep for 11 seconds 00000100:00100000:1.0:1789762825.092553:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074670080 00000100:00100000:1.0:1789762825.092558:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074670080 00000100:00100000:1.0:1789762825.092586:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 264 00000100:00100000:1.0:1789762825.092601:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074670080:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1789762825.092674:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074670080:12345-0@lo:1000:mdt00_001.0 Request processed in 74us (151us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762825.092681:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 264 00000100:00100000:3.0:1789762825.092687:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074670080:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.092721:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074670208:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.092742:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762825.092757:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f900 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.092764:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074670208 00000100:00100000:2.0:1789762825.092768:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074670208 00000100:00100000:2.0:1789762825.092792:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 265 00000100:00100000:2.0:1789762825.092800:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074670208:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762825.092849:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074670208:12345-0@lo:1000:mdt00_001.0 Request processed in 52us (106us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.092855:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 265 00000100:00100000:3.0:1789762825.092859:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074670208:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.092895:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074670336:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.092936:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762825.092953:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f900 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.092978:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074670336 00000100:00100000:2.0:1789762825.092984:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074670336 00000100:00100000:2.0:1789762825.093010:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 266 00000100:00100000:2.0:1789762825.093015:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074670336:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762825.093059:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074670336:12345-0@lo:1000:mdt00_001.0 Request processed in 44us (141us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.093065:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 266 00000100:00100000:3.0:1789762825.093083:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074670336:0@lo:1000:mdt00_001.0 00000040:00080000:3.0:1789762825.094532:0:6419:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].7, 984 off37976 00000040:00080000:3.0:1789762825.094702:0:6419:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].7, 1136 off38824 00000004:00080000:3.0:1789762825.094711:0:6419:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37d176fc00 with version 16 00000004:00080000:2.0:1789762825.094740:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 16 rpc_version 16 00000020:00100000:2.0:1789762825.094759:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1789762825.094765:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37d8c9c700 x1876702074670592/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2416/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762825.094777:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9c700 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074670592:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762825.094801:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762825.094816:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ee88d6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762825.094826:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074670592 00000100:00100000:3.0:1789762825.094831:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074670592 00000100:00100000:3.0:1789762825.094873:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 267 00000100:00100000:3.0:1789762825.094883:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7224:x1876702074670592:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1789762825.095560:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7224:x1876702074670592:12345-0@lo:1000:osp_up0-1.0 Request processed in 679us (756us total) trans 4294967385 rc 0/0 00000100:00100000:3.0:1789762825.095567:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 267 00000100:00100000:2.0:1789762825.095597:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9c700 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074670592:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1789762825.095627:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92e00/0xdf5a7ac59f175600 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.095636:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92e00/0xdf5a7ac59f175600 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.095645:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92e00/0xdf5a7ac59f175600 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762825.095654:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009f23255f reply state 00000000f8faae4c transno 4294967378 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92e00/0xdf5a7ac59f175600 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.095679:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92e00/0xdf5a7ac59f175600 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.095684:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762825.095686:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1789762825.095688:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762825.095692:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe:0x0]: returning size 16384 00000004:00080000:3.0:1789762825.095696:0:6419:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1789762825.095702:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.095709:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.095725:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.095742:0:6419:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1789762825.095746:0:6419:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1789762825.095750:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1755f9 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.095755:0:6419:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1755f9 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.095778:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762825.095786:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.095795:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93600 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:1.0:1789762825.095800:0:7217:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762825.095805:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13911:x1876702067360896:12345-192.168.203.37@tcp:36:mkdir.0 Request processed in 6663us (6760us total) trans 4294967378 rc 0/0 00000004:00010000:1.0:1789762825.095816:0:7217:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762825.095817:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 206 00010000:00010000:1.0:1789762825.095844:0:7217:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.095850:0:7217:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789762825.095874:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.095881:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d93600/0xdf5a7ac59f1755e4 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f1755eb expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762825.095884:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff882d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074670720:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762825.095897:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762825.095900:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762825.095902:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762825.095973:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762825.096008:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074670720 00000100:00100000:1.0:1789762825.096020:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074670720 00000100:00100000:1.0:1789762825.096056:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 32 00000100:00100000:1.0:1789762825.096063:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+9:3425:x1876702074670720:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762825.096071:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762825.096082:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f1755eb lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f1755e4 expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.096097:0:9878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762825.096099:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762825.096136:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3425:x1876702074670720:12345-0@lo:103:ldlm_bl.0 Request processed in 73us (161us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762825.096141:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 32 00000100:00100000:3.0:1789762825.096165:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff882d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074670720:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789762825.101526:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.101563:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067361152 00000100:00100000:0.0:1789762825.101569:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067361152 00000100:00100000:0.0:1789762825.101615:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 207 00000100:00100000:0.0:1789762825.101627:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:13912:x1876702067361152:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762825.101637:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762825.101648:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f175607 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6aa expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.101708:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f17560e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.101723:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f17560e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.101731:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f17560e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762825.101748:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762825.101751:0:11692:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762825.101763:0:11692:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f17560e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.101780:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f175607 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6aa expref: 7 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.101791:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f17560e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6aa expref: 6 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.101797:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762825.101799:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762825.101802:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94c00, rc 0) 00000100:00100000:0.0:1789762825.101834:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13912:x1876702067361152:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 213us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762825.101841:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 207 00000100:00100000:2.0:1789762825.102805:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.102842:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067361408 00000100:00100000:0.0:1789762825.102848:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067361408 00000100:00100000:0.0:1789762825.102880:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 208 00000100:00100000:0.0:1789762825.102887:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13912:x1876702067361408:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762825.102895:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762825.102903:0:6418:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1789762825.102914:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f17560e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6aa expref: 6 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.102952:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762825.102956:0:6418:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762825.102964:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94600/0xdf5a7ac59f175615 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6b1 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.103022:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f17561c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.103029:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f17561c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.103035:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f17561c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.103045:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.103051:0:6418:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f1755f9 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1755f2 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.103057:0:6418:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.103074:0:6418:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f1755f9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xdf5a7ac59f1755f2 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762825.103082:0:6418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6418:1876702074670848:0@lo:104: 00000100:00100000:0.0:1789762825.103114:0:6418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762825.103150:0:8068:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074670848 00000100:00100000:0.0:1789762825.103152:0:6418:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c9a9dc00 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.103162:0:8068:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074670848 00000100:00100000:2.0:1789762825.103206:0:8068:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 46 00000100:00100000:2.0:1789762825.103217:0:8068:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:6418:x1876702074670848:12345-0@lo:104: 00010000:00010000:2.0:1789762825.103226:0:8068:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f1755f9 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.103274:0:8068:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1789762825.103282:0:8068:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:6418:x1876702074670848:12345-0@lo:104: Request processed in 67us (166us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762825.103286:0:6418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6418:1876702074670848:0@lo:104: 00000100:00100000:2.0:1789762825.103299:0:8068:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 46 00010000:00010000:0.0:1789762825.103304:0:6418:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.103326:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762825.103333:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f1755f9 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.103341:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d92600 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:1.0:1789762825.103345:0:6408:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xdf5a7ac59f1755f9 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762825.103351:0:6408:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xdf5a7ac59f1755f9 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.103372:0:6408:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f1755f9 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.103376:0:6408:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789762825.103399:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f1755f9 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762825.103412:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff882680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074670976:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762825.103421:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37f7d92600/0xdf5a7ac59f1755f2 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f1755f9 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.103439:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762825.103441:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762825.103445:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762825.103448:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762825.103473:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074670976 00000100:00100000:1.0:1789762825.103478:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074670976 00010000:00010000:1.0:1789762825.103495:0:6405:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f1755f9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xdf5a7ac59f1755f2 expref: 8 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762825.103518:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 70 00000100:00100000:1.0:1789762825.103527:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c90050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3426:x1876702074670976:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762825.103535:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1789762825.112793:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967385 is committed 00010000:00010000:2.0:1789762825.114697:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f1755f9 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xdf5a7ac59f1755f2 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.114710:0:6405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743500) 00010000:00010000:2.0:1789762825.114714:0:6405:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.114720:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.114724:0:6405:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.114742:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762825.114762:0:6418:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.114771:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762825.114785:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c90050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074670976:12345-0@lo:103:ldlm_bl.0 Request processed in 11259us (11337us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.114802:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 70 00000100:00100000:3.0:1789762825.114826:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff882680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074670976:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762825.114940:0:6418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6418:1876702074671104:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1789762825.114976:0:6418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762825.115000:0:6418:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c9a9d9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.115001:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074671104 00000100:00100000:2.0:1789762825.115007:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074671104 00000100:00100000:2.0:1789762825.115042:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 268 00000100:00100000:2.0:1789762825.115051:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6418:x1876702074671104:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1789762825.115113:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6418:x1876702074671104:12345-0@lo:1000:mdt00_000.0 Request processed in 64us (138us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.115120:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 268 00000100:00100000:0.0:1789762825.115131:0:6418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6418:1876702074671104:0@lo:1000:mdt00_000.0 00010000:00010000:0.0:1789762825.115205:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f17562a lrc: 3/1,0 mode: --/PR res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115214:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f17562a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115220:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f17562a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115231:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f17561c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115236:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f17561c lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115249:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f17561c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115253:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f17561c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115258:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762825.115261:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f17561c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115266:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f17561c lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115274:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762825.115276:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762825.115279:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f17561c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115284:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f17561c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115293:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115297:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115301:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115305:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762825.115312:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115316:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115323:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743500) 00010000:00010000:0.0:1789762825.115325:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762825.115327:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115331:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f175623 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115358:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94600/0xdf5a7ac59f175615 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6b1 expref: 7 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115368:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f17562a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6b1 expref: 6 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.115373:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37e125ab00) 00010000:00010000:0.0:1789762825.115375:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762825.115376:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94c00, rc 0) 00000100:00100000:0.0:1789762825.115416:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13912:x1876702067361408:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 12527us (12609us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762825.115425:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 208 00000100:00100000:2.0:1789762825.116461:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.116492:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067361664 00000100:00100000:3.0:1789762825.116498:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067361664 00000100:00100000:3.0:1789762825.116542:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 209 00000100:00100000:3.0:1789762825.116552:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13912:x1876702067361664:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:3.0:1789762825.116564:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762825.116576:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175631 lrc: 2/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6b8 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762825.116632:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe:0x0]: returning size 16384 00010000:00010000:3.0:1789762825.116649:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175631 lrc: 1/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6b8 expref: 8 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.116673:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762825.116676:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762825.116680:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175631 lrc: 0/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6b8 expref: 8 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.116690:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d92a00, rc 0) 00000100:00100000:3.0:1789762825.116719:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13912:x1876702067361664:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 171us (262us total) trans 0 rc 301/301 00000100:00100000:3.0:1789762825.116724:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 209 00000100:00100000:2.0:1789762825.117824:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.118976:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067361792 00000100:00100000:2.0:1789762825.118982:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067361792 00000100:00100000:2.0:1789762825.119016:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 210 00000100:00100000:2.0:1789762825.119027:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13912:x1876702067361792:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762825.119035:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762825.119043:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af200/0xdf5a7ac59f175638 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6bf expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.119076:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af600/0xdf5a7ac59f17563f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.119084:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af600/0xdf5a7ac59f17563f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.119091:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af600/0xdf5a7ac59f17563f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762825.119106:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762825.119114:0:7744:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762825.119128:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af600/0xdf5a7ac59f17563f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.119143:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af200/0xdf5a7ac59f175638 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6bf expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.119152:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af600/0xdf5a7ac59f17563f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6bf expref: 7 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.119158:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762825.119160:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762825.119163:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d10af600, rc 0) 00000100:00100000:2.0:1789762825.119202:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13912:x1876702067361792:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 177us (1376us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.119211:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 210 00000100:00100000:1.0:1789762825.120149:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762825.120173:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067362048 00000100:00100000:1.0:1789762825.120177:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067362048 00000100:00100000:1.0:1789762825.120208:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 211 00000100:00100000:1.0:1789762825.120215:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13912:x1876702067362048:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:1.0:1789762825.120224:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762825.120234:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230000/0xdf5a7ac59f175646 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6c6 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120267:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f17564d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120276:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f17564d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120289:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f17564d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120303:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175654 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120308:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175654 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120313:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175654 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120442:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17565b lrc: 3/1,0 mode: --/PR res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120449:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17565b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120457:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17565b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120465:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f17564d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120470:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f17564d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120475:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f17564d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120479:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f17564d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120494:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762825.120498:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f17564d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120503:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f17564d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120511:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762825.120514:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762825.120517:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f17564d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120521:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231400/0xdf5a7ac59f17564d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120530:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175654 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120535:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175654 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120539:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175654 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120543:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762825.120545:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175654 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120549:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175654 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120565:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743300) 00010000:00010000:1.0:1789762825.120568:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762825.120570:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175654 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120574:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175654 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120594:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230000/0xdf5a7ac59f175646 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6c6 expref: 9 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120607:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17565b lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6c6 expref: 8 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.120612:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37e125ab00) 00010000:00010000:1.0:1789762825.120614:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789762825.120616:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d9230a00, rc 0) 00000100:00100000:1.0:1789762825.120645:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13912:x1876702067362048:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 431us (499us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762825.120650:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 211 00000100:00100000:1.0:1789762825.121684:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.121718:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067362432 00000100:00100000:2.0:1789762825.121723:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067362432 00000100:00100000:1.0:1789762825.121729:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.121761:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 33 00000100:00100000:3.0:1789762825.121768:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067362304 00000100:00100000:2.0:1789762825.121769:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067362432:12345-192.168.203.37@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1789762825.121777:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067362304 00010000:00010000:2.0:1789762825.121780:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762825.121791:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17565b lrc: 0/0,0 mode: --/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6c6 expref: 8 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.121801:0:8070:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37e125ab00) 00010000:00010000:2.0:1789762825.121804:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762825.121829:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067362432:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 61us (146us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.121843:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 33 00000100:00100000:3.0:1789762825.121902:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 212 00000100:00100000:3.0:1789762825.121979:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13912:x1876702067362304:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:3.0:1789762825.121989:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762825.121998:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175662 lrc: 2/0,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6cd expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762825.122033:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe:0x0]: returning size 16384 00010000:00010000:3.0:1789762825.122097:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175662 lrc: 1/0,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6cd expref: 9 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.122105:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762825.122113:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762825.122117:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175662 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6cd expref: 9 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.122125:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d92a00, rc 0) 00000100:00100000:3.0:1789762825.122159:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13912:x1876702067362304:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 247us (428us total) trans 4294967379 rc 301/301 00000100:00100000:3.0:1789762825.122169:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 212 00000100:00100000:1.0:1789762825.123586:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.123614:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067362688 00000100:00100000:2.0:1789762825.123618:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067362688 00000100:00100000:2.0:1789762825.123644:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 213 00000100:00100000:2.0:1789762825.123652:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13912:x1876702067362688:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:2.0:1789762825.123695:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aee00/0xdf5a7ac59f175670 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.123704:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aee00/0xdf5a7ac59f175670 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.123710:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aee00/0xdf5a7ac59f175670 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762825.123826:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003f3bc2c9 reply state 00000000a8f9a5f3 transno 4294967380 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aee00/0xdf5a7ac59f175670 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.123835:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aee00/0xdf5a7ac59f175670 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.123840:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:2.0:1789762825.123842:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:2.0:1789762825.123844:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762825.123848:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe:0x0]: returning size 16384 00000100:00100000:2.0:1789762825.123881:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:13912:x1876702067362688:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 231us (295us total) trans 4294967380 rc 0/0 00000100:00100000:2.0:1789762825.123887:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 213 00000100:00100000:1.0:1789762825.124557:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.124592:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067362944 00000100:00100000:2.0:1789762825.124596:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067362944 00000100:00100000:2.0:1789762825.124629:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 54 00000100:00100000:2.0:1789762825.124635:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:13912:x1876702067362944:12345-192.168.203.37@tcp:35:lfs.0 00000100:00100000:2.0:1789762825.124737:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:13912:x1876702067362944:12345-192.168.203.37@tcp:35:lfs.0 Request processed in 102us (181us total) trans 4294967381 rc 0/0 00000100:00100000:2.0:1789762825.124742:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 54 00000100:00100000:1.0:1789762825.129042:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.129073:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067363072 00000100:00100000:2.0:1789762825.129078:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067363072 00000100:00100000:2.0:1789762825.129103:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 214 00000100:00100000:2.0:1789762825.129108:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:13913:x1876702067363072:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:2.0:1789762825.129118:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762825.129127:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af200/0xdf5a7ac59f175677 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6d4 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.129162:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af400/0xdf5a7ac59f17567e lrc: 3/1,0 mode: --/PR res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.129173:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af400/0xdf5a7ac59f17567e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.129182:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af400/0xdf5a7ac59f17567e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762825.129201:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe:0x0]: returning size 16384 00000004:00200000:2.0:1789762825.129205:0:6420:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762825.129217:0:6420:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af400/0xdf5a7ac59f17567e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.129232:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af200/0xdf5a7ac59f175677 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6d4 expref: 13 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.129255:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af400/0xdf5a7ac59f17567e lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6d4 expref: 12 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.129260:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:2.0:1789762825.129263:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762825.129267:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d10af400, rc 0) 00000100:00100000:2.0:1789762825.129300:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:13913:x1876702067363072:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 190us (258us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.129308:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 214 00000100:00100000:1.0:1789762825.130216:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.130251:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067363328 00000100:00100000:3.0:1789762825.130257:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067363328 00000100:00100000:3.0:1789762825.130284:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 215 00000100:00100000:3.0:1789762825.130290:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:13913:x1876702067363328:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:3.0:1789762825.130310:0:13389:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762825.130320:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af400/0xdf5a7ac59f17567e lrc: 0/0,0 mode: --/PR res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6d4 expref: 12 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.130331:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762825.130333:0:13389:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762825.130343:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175685 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.130350:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175685 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.130355:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175685 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762825.130445:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008480d740 reply state 0000000048ee691b transno 4294967382 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175685 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.130463:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175685 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.130468:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762825.130469:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:3.0:1789762825.130471:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762825.130476:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe:0x0]: returning size 16384 00000100:00100000:3.0:1789762825.130514:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:13913:x1876702067363328:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 224us (297us total) trans 4294967382 rc 0/0 00000100:00100000:3.0:1789762825.130520:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 215 00000100:00100000:1.0:1789762825.144278:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.144334:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067363456 00000100:00100000:2.0:1789762825.144339:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067363456 00000100:00100000:2.0:1789762825.144369:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 216 00000100:00100000:2.0:1789762825.144379:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13917:x1876702067363456:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762825.144391:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762825.144402:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af200/0xdf5a7ac59f17568c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6db expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144442:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f175693 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144451:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f175693 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144457:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f175693 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144465:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae800/0xdf5a7ac59f17569a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144477:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae800/0xdf5a7ac59f17569a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144481:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae800/0xdf5a7ac59f17569a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144604:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae200/0xdf5a7ac59f1756a1 lrc: 3/1,0 mode: --/PR res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144611:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae200/0xdf5a7ac59f1756a1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144616:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae200/0xdf5a7ac59f1756a1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762825.144641:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae200/0xdf5a7ac59f1756a1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144650:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f175693 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144668:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f175693 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144673:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f175693 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144678:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f175693 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144682:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d10afa00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762825.144684:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f175693 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144695:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f175693 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144703:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762825.144705:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762825.144708:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f175693 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144712:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10afa00/0xdf5a7ac59f175693 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144719:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae800/0xdf5a7ac59f17569a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144723:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae800/0xdf5a7ac59f17569a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144726:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae800/0xdf5a7ac59f17569a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144730:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d10ae800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762825.144732:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae800/0xdf5a7ac59f17569a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144735:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae800/0xdf5a7ac59f17569a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144739:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee100) 00010000:00010000:2.0:1789762825.144741:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762825.144743:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae800/0xdf5a7ac59f17569a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144753:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae800/0xdf5a7ac59f17569a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144771:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af200/0xdf5a7ac59f17568c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6db expref: 9 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144778:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae200/0xdf5a7ac59f1756a1 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6db expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.144783:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37e125ab00) 00010000:00010000:2.0:1789762825.144785:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762825.144787:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d10ae200, rc 0) 00000100:00100000:2.0:1789762825.144815:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13917:x1876702067363456:12345-192.168.203.37@tcp:101:dd.0 Request processed in 439us (540us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.144820:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 216 00000100:00100000:1.0:1789762825.145872:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762825.145896:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.145910:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067363584 00000100:00100000:0.0:1789762825.145936:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067363584 00000100:00100000:2.0:1789762825.145955:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067363712 00000100:00100000:2.0:1789762825.145961:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067363712 00000100:00100000:0.0:1789762825.145979:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 217 00000100:00100000:0.0:1789762825.145988:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:13917:x1876702067363584:12345-192.168.203.37@tcp:101:dd.0 00000100:00100000:2.0:1789762825.146004:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 34 00010000:00010000:0.0:1789762825.146005:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1789762825.146017:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067363712:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762825.146018:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94600/0xdf5a7ac59f1756a8 lrc: 2/0,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6e2 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.146029:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762825.146040:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10ae200/0xdf5a7ac59f1756a1 lrc: 0/0,0 mode: --/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6db expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.146049:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37e125ab00) 00010000:00010000:0.0:1789762825.146052:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f1756af lrc: 3/1,0 mode: --/PR res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.146052:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762825.146062:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f1756af lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.146069:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f1756af lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762825.146078:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067363712:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 65us (180us total) trans 0 rc 0/0 00000004:00200000:0.0:1789762825.146084:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe:0x0]: returning size 16384 00000004:00200000:0.0:1789762825.146090:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000100:00100000:2.0:1789762825.146095:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 34 00010000:00010000:0.0:1789762825.146112:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94600/0xdf5a7ac59f1756a8 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6e2 expref: 15 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.146125:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f1756af lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6e2 expref: 14 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.146131:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:0.0:1789762825.146133:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1789762825.146135:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94200, rc 0) 00000100:00100000:0.0:1789762825.146164:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13917:x1876702067363584:12345-192.168.203.37@tcp:101:dd.0 Request processed in 179us (294us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762825.146169:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 217 00000100:00100000:1.0:1789762825.147032:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.147060:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067363968 00000100:00100000:2.0:1789762825.147064:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067363968 00000100:00100000:2.0:1789762825.147088:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 218 00000100:00100000:2.0:1789762825.147093:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13917:x1876702067363968:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762825.147100:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762825.147107:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10afc00/0xdf5a7ac59f1756b6 lrc: 2/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6e9 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.147160:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aea00/0xdf5a7ac59f1756bd lrc: 3/0,1 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.147170:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aea00/0xdf5a7ac59f1756bd lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.147179:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aea00/0xdf5a7ac59f1756bd lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.147244:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10ae400/0xdf5a7ac59f1756c4 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xe:0x0].0x73550ecb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.147250:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10ae400/0xdf5a7ac59f1756c4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.147256:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10ae400/0xdf5a7ac59f1756c4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762825.147351:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074671232:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762825.147390:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762825.147427:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ee88d780 going to sleep for 11 seconds 00000100:00100000:3.0:1789762825.147436:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074671232 00000100:00100000:3.0:1789762825.147442:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074671232 00000100:00100000:3.0:1789762825.147474:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 269 00000100:00100000:3.0:1789762825.147481:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:6419:x1876702074671232:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762825.147536:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:6419:x1876702074671232:12345-0@lo:1000:mdt00_001.0 Request processed in 56us (146us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762825.147543:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 269 00000100:00100000:2.0:1789762825.147555:0:6419:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967385 gen 2 00000004:00080000:2.0:1789762825.147560:0:6419:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967385 committed transno 4294967385 00000100:00100000:2.0:1789762825.147578:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074671232:0@lo:1000:mdt00_001.0 00000004:00080000:2.0:1789762825.147824:0:6419:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x8:0x0], index 0: 0 00000004:00080000:2.0:1789762825.147837:0:6419:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x8:0x0], index 1: 0 00010000:00010000:2.0:1789762825.148013:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af000/0xdf5a7ac59f1756cb lrc: 3/1,0 mode: --/PR res: [0x240000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148023:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af000/0xdf5a7ac59f1756cb lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148033:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af000/0xdf5a7ac59f1756cb lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148066:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aea00/0xdf5a7ac59f1756bd lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148074:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aea00/0xdf5a7ac59f1756bd lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148081:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aea00/0xdf5a7ac59f1756bd lrc: 3/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148087:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aea00/0xdf5a7ac59f1756bd lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148093:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d10aea00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762825.148097:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aea00/0xdf5a7ac59f1756bd lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148105:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aea00/0xdf5a7ac59f1756bd lrc: 5/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148122:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:2.0:1789762825.148125:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762825.148128:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aea00/0xdf5a7ac59f1756bd lrc: 3/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148135:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aea00/0xdf5a7ac59f1756bd lrc: 0/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762825.148146:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000624f9f1a reply state 000000008aa9e9a6 transno 4294967383 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10ae400/0xdf5a7ac59f1756c4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148156:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10ae400/0xdf5a7ac59f1756c4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148162:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x73550ecb (ffff9c37d3beee00) 00010000:00010000:2.0:1789762825.148172:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762825.148175:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762825.148198:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10afc00/0xdf5a7ac59f1756b6 lrc: 0/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6e9 expref: 17 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148213:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af000/0xdf5a7ac59f1756cb lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6e9 expref: 16 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.148221:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xf:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762825.148223:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762825.148224:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d10af000, rc 0) 00000100:00100000:2.0:1789762825.148260:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:13917:x1876702067363968:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1165us (1228us total) trans 4294967383 rc 0/0 00000100:00100000:2.0:1789762825.148267:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 218 00000100:00100000:1.0:1789762825.151558:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.151704:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067364096 00000100:00100000:2.0:1789762825.151710:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067364096 00000100:00080000:3.0:1789762825.151754:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762825.151781:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 133 00000100:00100000:2.0:1789762825.151793:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:13917:x1876702067364096:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762825.151808:0:8052:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762825.151821:0:8052:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002a8b8c9b/0xdf5a7ac59f1756d9 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6f0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762825.151841:0:8052:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x8:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789762825.151856:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x8:0x0].0x0 since=298s v=(0 0) 00010000:00010000:2.0:1789762825.151860:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762825.151868:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002a8b8c9b/0xdf5a7ac59f1756d9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6f0 expref: 6 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.151880:0:8052:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002a8b8c9b/0xdf5a7ac59f1756d9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6f0 expref: 6 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.151889:0:8052:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d10af200, rc 0) 00000100:00100000:2.0:1789762825.151951:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13917:x1876702067364096:12345-192.168.203.37@tcp:101:dd.0 Request processed in 159us (394us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.151960:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 133 00000100:00100000:1.0:1789762825.159720:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.159755:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067364224 00000100:00100000:0.0:1789762825.159760:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067364224 00002000:00100000:0.0:1789762825.159795:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x8:0x0] (0->1048575) req@ffff9c37c94a3100 x1876702067364224/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.159803:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a3100 with x1876702067364224 ext(0->1048575) 00002000:00010000:0.0:1789762825.159806:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c94a3100 00000100:00100000:0.0:1789762825.159819:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263466051895 time 263468716476 state 0x0 ref 1 00000100:00100000:0.0:1789762825.159825:0:8059:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 5: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263466051895 time 263468716476 state 0x0 ref 1 00000100:00100000:0.0:1789762825.159839:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 3 deadline 298456916044 time 298406916044 state 0x1 ref 1 00000100:00100000:0.0:1789762825.159860:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 2 deadline 298456916044 time 298406938023 state 0x0 ref 1 00000100:00100000:0.0:1789762825.159868:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067364224:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762825.165301:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067364224:12345-192.168.203.37@tcp:4:dd.0 Request processed in 5431us (5581us total) trans 4294967675 rc 0/0 00000100:00100000:0.0:1789762825.165309:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 231 00000100:00100000:0.0:1789762825.165315:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 2 deadline 298456916044 time 298406938023 state 0x2 ref 0 00002000:00100000:0.0:1789762825.165324:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x8:0x0] (0->1048575) req@ffff9c37c94a3100 x1876702067364224/t4294967675(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.165332:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a3100 with x1876702067364224 ext(0->1048575) 00002000:00010000:0.0:1789762825.165335:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c94a3100 00000100:00100000:1.0:1789762825.166723:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.166781:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067364352 00000100:00100000:2.0:1789762825.166788:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067364352 00002000:00100000:2.0:1789762825.166810:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x8:0x0] (1048576->2097151) req@ffff9c37c94a0000 x1876702067364352/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762825.166817:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a0000 with x1876702067364352 ext(1048576->2097151) 00002000:00010000:2.0:1789762825.166821:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c94a0000 00000100:00100000:2.0:1789762825.166835:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 2 deadline 298456916044 time 298406938023 state 0x0 ref 1 00000100:00100000:2.0:1789762825.166843:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 2 deadline 298456938023 time 298406938023 state 0x1 ref 1 00000100:00100000:2.0:1789762825.166864:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 1 deadline 298456938023 time 298413942589 state 0x0 ref 1 00000100:00100000:2.0:1789762825.166873:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067364352:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762825.168974:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2397:x1876702067364352:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2100us (2250us total) trans 4294967676 rc 0/0 00000100:00100000:3.0:1789762825.168982:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 232 00000100:00100000:3.0:1789762825.168988:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 1 deadline 298456938023 time 298413942589 state 0x2 ref 0 00002000:00100000:3.0:1789762825.168996:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x8:0x0] (1048576->2097151) req@ffff9c37c94a0000 x1876702067364352/t4294967676(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.169004:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a0000 with x1876702067364352 ext(1048576->2097151) 00002000:00010000:3.0:1789762825.169013:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c94a0000 00000100:00100000:1.0:1789762825.174271:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.174347:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067364480 00000100:00100000:2.0:1789762825.174353:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067364480 00002000:00100000:2.0:1789762825.174373:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x8:0x0] (2097152->3145727) req@ffff9c37c94a0a80 x1876702067364480/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762825.174384:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a0a80 with x1876702067364480 ext(2097152->3145727) 00002000:00010000:2.0:1789762825.174388:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c94a0a80 00000100:00100000:2.0:1789762825.174399:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 1 deadline 298456938023 time 298413942589 state 0x0 ref 1 00000100:00100000:2.0:1789762825.174410:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 1 deadline 298463942589 time 298413942589 state 0x1 ref 1 00000100:00100000:2.0:1789762825.174439:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298463942589 time 298421515729 state 0x0 ref 1 00000100:00100000:2.0:1789762825.174449:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2398:x1876702067364480:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762825.176592:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:2398:x1876702067364480:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2143us (2321us total) trans 4294967677 rc 0/0 00000100:00100000:0.0:1789762825.176604:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 233 00000100:00100000:0.0:1789762825.176610:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298463942589 time 298421515729 state 0x2 ref 0 00002000:00100000:0.0:1789762825.176618:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x8:0x0] (2097152->3145727) req@ffff9c37c94a0a80 x1876702067364480/t4294967677(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.176625:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a0a80 with x1876702067364480 ext(2097152->3145727) 00002000:00010000:0.0:1789762825.176628:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c94a0a80 00000100:00100000:1.0:1789762825.182410:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.182469:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067364608 00000100:00100000:2.0:1789762825.182474:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067364608 00002000:00100000:2.0:1789762825.182496:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0x8:0x0] (3145728->4194303) req@ffff9c37c94a1500 x1876702067364608/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762825.182514:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a1500 with x1876702067364608 ext(3145728->4194303) 00002000:00010000:2.0:1789762825.182518:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c94a1500 00000100:00100000:2.0:1789762825.182526:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298463942589 time 298421515729 state 0x0 ref 1 00000100:00100000:2.0:1789762825.182535:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298471515729 time 298421515729 state 0x1 ref 1 00000100:00100000:1.0:1789762825.182551:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.182597:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067364736 00000100:00100000:0.0:1789762825.182602:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067364736 00000100:00100000:0.0:1789762825.182624:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 134 00000100:00100000:0.0:1789762825.182628:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0+-99:2394:x1876702067364736:12345-192.168.203.37@tcp:8:dd.0 00010000:00080000:0.0:1789762825.182650:0:9037:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp t0 exp 0000000000000000 cur 298 last 0 00000020:00000080:0.0:1789762825.182685:0:9037:0:(genops.c:1385:class_connect()) connect: client fa6fc127-0d58-47ca-82d1-2c8ad7b086c1, cookie 0xdf5a7ac59f1756e0 00002000:00100000:0.0:1789762825.182694:0:9037:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli fa6fc127-0d58-47ca-82d1-2c8ad7b086c1/ffff9c36cba8e000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:0.0:1789762825.182707:0:9037:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:0.0:1789762825.182831:0:9037:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:0.0:1789762825.182854:0:9037:0:(import.c:65:import_set_state_nolock()) ffff9c36cba88800 : changing import state from RECOVER to FULL 00000100:00100000:0.0:1789762825.182885:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2394:x1876702067364736:12345-192.168.203.37@tcp:8:dd.0 Request processed in 256us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762825.182891:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 134 00000100:00100000:1.0:1789762825.183585:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.183616:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067364864 00000100:00100000:3.0:1789762825.183621:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067364864 00000100:00100000:3.0:1789762825.183667:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 135 00000100:00100000:3.0:1789762825.183677:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13917:x1876702067364864:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762825.183687:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:2.0:1789762825.183689:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1789762825.183701:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ee1ee3ef/0xdf5a7ac59f1756e7 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6f7 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762825.183720:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x8:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762825.183732:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x8:0x0].0x0 since=298s v=(0 0) 00010000:00010000:3.0:1789762825.183736:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762825.183743:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ee1ee3ef/0xdf5a7ac59f1756e7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6f7 expref: 7 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.183754:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ee1ee3ef/0xdf5a7ac59f1756e7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6f7 expref: 7 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.183764:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d92400, rc 0) 00000100:00100000:3.0:1789762825.183790:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13917:x1876702067364864:12345-192.168.203.37@tcp:101:dd.0 Request processed in 116us (207us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762825.183795:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 135 00000100:00100000:1.0:1789762825.191102:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.191138:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067364992 00000100:00100000:0.0:1789762825.191142:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067364992 00002000:00100000:0.0:1789762825.191158:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x8:0x0] (0->1048575) req@ffff9c37c94a1f80 x1876702067364992/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.191181:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a1f80 with x1876702067364992 ext(0->1048575) 00002000:00010000:0.0:1789762825.191185:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c94a1f80 00000100:00100000:0.0:1789762825.191198:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298471515729 time 298421515729 state 0x1 ref 2 00000100:00100000:1.0:1789762825.197945:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.197977:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067365120 00000100:00100000:3.0:1789762825.197983:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067365120 00002000:00100000:3.0:1789762825.198001:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x8:0x0] (1048576->2097151) req@ffff9c37c94a3b80 x1876702067365120/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.198008:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a3b80 with x1876702067365120 ext(1048576->2097151) 00002000:00010000:3.0:1789762825.198017:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c94a3b80 00000100:00100000:3.0:1789762825.198027:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298471515729 time 298421515729 state 0x1 ref 3 00000100:00100000:0.0:1789762825.224515:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298521582401 time 298471582401 state 0x1 ref 3 00000100:00100000:0.0:1789762825.224534:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:2397:x1876702067364608:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762825.227277:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:2397:x1876702067364608:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2748us (44867us total) trans 4294967678 rc 0/0 00000100:00100000:0.0:1789762825.227284:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 234 00002000:00100000:0.0:1789762825.227292:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x8:0x0] (3145728->4194303) req@ffff9c37c94a1500 x1876702067364608/t4294967678(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.227299:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a1500 with x1876702067364608 ext(3145728->4194303) 00002000:00010000:0.0:1789762825.227304:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c94a1500 00000100:00100000:2.0:1789762825.274593:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298571659290 time 298521659290 state 0x1 ref 2 00000100:00100000:2.0:1789762825.274617:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067364992:12345-192.168.203.37@tcp:4:dd.0 00000001:00100000:0.0:1789762825.285536:0:9000:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000001:00080000:0.0:1789762825.295256:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967364 is committed 00000100:00100000:2.0:1789762825.295328:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067364992:12345-192.168.203.37@tcp:4:dd.0 Request processed in 20718us (104222us total) trans 4294967364 rc 0/0 00000100:00100000:2.0:1789762825.295337:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 235 00002000:00100000:2.0:1789762825.295348:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x8:0x0] (0->1048575) req@ffff9c37c94a1f80 x1876702067364992/t4294967364(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762825.295356:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a1f80 with x1876702067364992 ext(0->1048575) 00002000:00010000:2.0:1789762825.295360:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c94a1f80 00000100:00100000:1.0:1789762825.297536:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.297586:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067365248 00000100:00100000:3.0:1789762825.297596:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067365248 00002000:00100000:3.0:1789762825.297631:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0x8:0x0] (2097152->3145727) req@ffff9c37c94a1180 x1876702067365248/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.297642:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a1180 with x1876702067365248 ext(2097152->3145727) 00002000:00010000:3.0:1789762825.297647:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c94a1180 00000100:00100000:3.0:1789762825.297682:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298571659290 time 298521659290 state 0x1 ref 2 00000100:00100000:2.0:1789762825.303933:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.303985:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067365376 00000100:00100000:0.0:1789762825.303991:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067365376 00002000:00100000:0.0:1789762825.304010:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0x8:0x0] (3145728->4194303) req@ffff9c37d8c9ce00 x1876702067365376/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.304018:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c9ce00 with x1876702067365376 ext(3145728->4194303) 00002000:00010000:0.0:1789762825.304021:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8c9ce00 00000100:00100000:0.0:1789762825.304030:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298571659290 time 298521659290 state 0x1 ref 3 00000100:00100000:1.0:1789762825.310258:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.310309:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067365504 00000100:00100000:0.0:1789762825.310314:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067365504 00002000:00100000:0.0:1789762825.310335:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x8:0x0] (4194304->5242879) req@ffff9c37c94a2300 x1876702067365504/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.310343:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a2300 with x1876702067365504 ext(4194304->5242879) 00002000:00010000:0.0:1789762825.310347:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c94a2300 00000100:00100000:0.0:1789762825.310355:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298571659290 time 298521659290 state 0x1 ref 4 00000100:00100000:1.0:1789762825.316601:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.316799:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067365632 00000100:00100000:2.0:1789762825.316805:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067365632 00002000:00100000:2.0:1789762825.316823:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0x8:0x0] (5242880->6291455) req@ffff9c37e3bde680 x1876702067365632/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762825.316831:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bde680 with x1876702067365632 ext(5242880->6291455) 00002000:00010000:2.0:1789762825.316840:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e3bde680 00000100:00100000:2.0:1789762825.316850:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298571659290 time 298521659290 state 0x1 ref 5 00000100:00100000:2.0:1789762825.323448:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.323494:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067365760 00000100:00100000:0.0:1789762825.323499:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067365760 00002000:00100000:0.0:1789762825.323519:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x8:0x0] (6291456->7340031) req@ffff9c37d8c9c700 x1876702067365760/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.323526:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c9c700 with x1876702067365760 ext(6291456->7340031) 00002000:00010000:0.0:1789762825.323529:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8c9c700 00000100:00100000:0.0:1789762825.323538:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298571659290 time 298521659290 state 0x1 ref 6 00000100:00100000:3.0:1789762825.324671:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298621732013 time 298571732013 state 0x1 ref 6 00000100:00100000:3.0:1789762825.324686:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067365120:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762825.327339:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2397:x1876702067365120:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2655us (129392us total) trans 4294967365 rc 0/0 00000100:00100000:3.0:1789762825.327348:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 236 00002000:00100000:3.0:1789762825.327356:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x8:0x0] (1048576->2097151) req@ffff9c37c94a3b80 x1876702067365120/t4294967365(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.327366:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a3b80 with x1876702067365120 ext(1048576->2097151) 00002000:00010000:3.0:1789762825.327370:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c94a3b80 00000100:00100000:1.0:1789762825.331576:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.331621:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067365888 00000100:00100000:0.0:1789762825.331626:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067365888 00002000:00100000:0.0:1789762825.331642:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x8:0x0] (7340032->8388607) req@ffff9c37e3bdc000 x1876702067365888/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.331649:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bdc000 with x1876702067365888 ext(7340032->8388607) 00002000:00010000:0.0:1789762825.331652:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e3bdc000 00000100:00100000:0.0:1789762825.331675:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298621732013 time 298571732013 state 0x1 ref 6 00000100:00100000:2.0:1789762825.337952:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.337988:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067366016 00000100:00100000:3.0:1789762825.337994:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067366016 00002000:00100000:3.0:1789762825.338016:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x8:0x0] (4194304->5242879) req@ffff9c37d8c9f800 x1876702067366016/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.338038:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c9f800 with x1876702067366016 ext(4194304->5242879) 00002000:00010000:3.0:1789762825.338042:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8c9f800 00000100:00100000:3.0:1789762825.338055:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298621732013 time 298571732013 state 0x1 ref 7 00000100:00100000:1.0:1789762825.344101:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.344144:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067366144 00000100:00100000:0.0:1789762825.344150:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067366144 00002000:00100000:0.0:1789762825.344173:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x8:0x0] (5242880->6291455) req@ffff9c37e3bdfb80 x1876702067366144/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.344183:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bdfb80 with x1876702067366144 ext(5242880->6291455) 00002000:00010000:0.0:1789762825.344187:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3bdfb80 00000100:00100000:0.0:1789762825.344196:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298621732013 time 298571732013 state 0x1 ref 8 00000100:00100000:3.0:1789762825.374717:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298671786044 time 298621786044 state 0x1 ref 8 00000100:00100000:3.0:1789762825.374736:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:2398:x1876702067365248:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762825.377521:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:2398:x1876702067365248:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2787us (79983us total) trans 4294967366 rc 0/0 00000100:00100000:3.0:1789762825.377531:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 237 00002000:00100000:3.0:1789762825.377541:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x8:0x0] (2097152->3145727) req@ffff9c37c94a1180 x1876702067365248/t4294967366(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.377551:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a1180 with x1876702067365248 ext(2097152->3145727) 00002000:00010000:3.0:1789762825.377556:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c94a1180 00000100:00100000:1.0:1789762825.379441:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.379483:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067366272 00000100:00100000:0.0:1789762825.379489:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067366272 00002000:00100000:0.0:1789762825.379511:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0x8:0x0] (6291456->7340031) req@ffff9c37e3bdf480 x1876702067366272/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.379518:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bdf480 with x1876702067366272 ext(6291456->7340031) 00002000:00010000:0.0:1789762825.379521:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3bdf480 00000100:00100000:0.0:1789762825.379531:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298671786044 time 298621786044 state 0x1 ref 8 00000100:00100000:2.0:1789762825.385717:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.385761:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.385770:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067366400 00000100:00100000:3.0:1789762825.385777:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067366400 00002000:00100000:3.0:1789762825.385802:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x8:0x0] (7340032->8388607) req@ffff9c37f6089c00 x1876702067366400/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1789762825.385806:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067366656 00000100:00100000:1.0:1789762825.385813:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067366656 00002000:00100000:3.0:1789762825.385816:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f6089c00 with x1876702067366400 ext(7340032->8388607) 00002000:00010000:3.0:1789762825.385822:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f6089c00 00000100:00100000:3.0:1789762825.385834:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298671786044 time 298621786044 state 0x1 ref 9 00000100:00100000:1.0:1789762825.385851:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 55 00000100:00100000:1.0:1789762825.385861:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:13917:x1876702067366656:12345-192.168.203.37@tcp:35:dd.0 00000001:00080000:1.0:1789762825.386040:0:6422:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967384 00000100:00100000:1.0:1789762825.386103:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:13917:x1876702067366656:12345-192.168.203.37@tcp:35:dd.0 Request processed in 245us (342us total) trans 4294967385 rc 0/0 00000100:00100000:1.0:1789762825.386110:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 55 00000020:00080000:1.0:1789762825.386113:0:6422:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1789762825.416465:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.416508:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067366912 00000100:00100000:0.0:1789762825.416515:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067366912 00000100:00100000:0.0:1789762825.416556:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 219 00000100:00100000:0.0:1789762825.416567:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:13927:x1876702067366912:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762825.416587:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762825.416607:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94600/0xdf5a7ac59f1756ee lrc: 2/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c6fe expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.416713:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f1756f5 lrc: 3/0,1 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.416733:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f1756f5 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.416742:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f1756f5 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.416801:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f1756fc lrc: 3/0,1 mode: --/PW res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.416806:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f1756fc lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.416811:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f1756fc lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762825.417096:0:11692:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x9:0x0], index 1: 0 00000004:00080000:0.0:1789762825.417105:0:11692:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x9:0x0], index 0: 0 00010000:00010000:0.0:1789762825.417233:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f175703 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417241:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f175703 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417247:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f175703 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417269:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f1756f5 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417275:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f1756f5 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417288:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f1756f5 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417293:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762825.417296:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f1756f5 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417301:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f1756f5 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417310:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:0.0:1789762825.417313:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762825.417317:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f1756f5 lrc: 2/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417321:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95a00/0xdf5a7ac59f1756f5 lrc: 0/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762825.417329:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006d1cd081 reply state 000000007f2c44bb transno 4294967386 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f1756fc lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417336:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f1756fc lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417340:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x2b0e463d (ffff9c37e125ad00) 00010000:00010000:0.0:1789762825.417342:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762825.417344:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762825.417364:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94600/0xdf5a7ac59f1756ee lrc: 0/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6fe expref: 22 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417383:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f175703 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6fe expref: 21 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.417398:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x10:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:0.0:1789762825.417401:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762825.417403:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94a00, rc 0) 00000100:00100000:0.0:1789762825.417450:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:13927:x1876702067366912:12345-192.168.203.37@tcp:101:dd.0 Request processed in 885us (988us total) trans 4294967386 rc 0/0 00000100:00100000:0.0:1789762825.417459:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 219 00000100:00100000:2.0:1789762825.422977:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.423024:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067367040 00000100:00100000:3.0:1789762825.423036:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067367040 00000100:00100000:3.0:1789762825.423134:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 136 00000100:00100000:3.0:1789762825.423146:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:13928:x1876702067367040:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762825.423156:0:8052:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762825.423170:0:8052:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c705 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762825.423191:0:8052:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x9:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762825.423205:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=298s v=(0 0) 00010000:00010000:3.0:1789762825.423210:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(0 0) 00000100:00100000:1.0:1789762825.423220:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:3.0:1789762825.423221:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 16 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.423239:0:8052:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 16 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.423251:0:8052:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d93a00, rc 0) 00000100:00100000:0.0:1789762825.423259:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067367168 00000100:00100000:0.0:1789762825.423264:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067367168 00000100:00100000:0.0:1789762825.423292:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 137 00000100:00100000:0.0:1789762825.423302:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:13927:x1876702067367168:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762825.423313:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762825.423323:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175718 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c70c expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762825.423337:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13928:x1876702067367040:12345-192.168.203.37@tcp:101:dd.0 Request processed in 192us (357us total) trans 0 rc 0/0 00010000:00010000:0.0:1789762825.423345:0:9037:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 18 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762825.423351:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 136 00010000:00010000:0.0:1789762825.423362:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762825.423366:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762825.423373:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175718 lrc: 4/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c70c expref: 17 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.423410:0:9037:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 17 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.423439:0:9037:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 17 pid: 8052 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762825.423453:0:9037:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9037:1876702074671360:192.168.203.37@tcp:104: 00000100:00100000:0.0:1789762825.423519:0:9037:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c9a9d000 going to sleep for 11 seconds 00000100:00100000:0.0:1789762825.424240:0:9037:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9037:1876702074671360:192.168.203.37@tcp:104: 00010000:00010000:0.0:1789762825.424255:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175718 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c70c expref: 17 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.424263:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94600, rc 0) 00000100:00100000:0.0:1789762825.424290:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:13927:x1876702067367168:12345-192.168.203.37@tcp:101:dd.0 Request processed in 990us (1073us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762825.424298:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 137 00000100:00100000:0.0:1789762825.424766:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298721840208 time 298671840208 state 0x1 ref 9 00000100:00100000:0.0:1789762825.424777:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+16:2398:x1876702067365376:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762825.424913:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.424963:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067367296 00000100:00100000:3.0:1789762825.424969:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067367296 00000100:00100000:2.0:1789762825.425009:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 138 00000100:00100000:2.0:1789762825.425017:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13929:x1876702067367296:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762825.425023:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762825.425031:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cd624d43/0xdf5a7ac59f17571f lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c713 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.425050:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789762825.425054:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(2 0) 00010000:00010000:2.0:1789762825.425057:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cd624d43/0xdf5a7ac59f17571f lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c713 expref: 19 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.425065:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cd624d43/0xdf5a7ac59f17571f lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c713 expref: 19 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.425072:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d10ae000, rc 0) 00000100:00100000:2.0:1789762825.425096:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:13929:x1876702067367296:12345-192.168.203.37@tcp:101:dd.0 Request processed in 81us (184us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.425109:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 138 00000100:00100000:0.0:1789762825.427230:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:2398:x1876702067365376:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2452us (123312us total) trans 4294967367 rc 0/0 00000100:00100000:0.0:1789762825.427241:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 238 00002000:00100000:0.0:1789762825.427250:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x8:0x0] (3145728->4194303) req@ffff9c37d8c9ce00 x1876702067365376/t4294967367(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.427261:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c9ce00 with x1876702067365376 ext(3145728->4194303) 00002000:00010000:0.0:1789762825.427273:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8c9ce00 00000100:00100000:1.0:1789762825.430510:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.430588:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067367424 00000100:00100000:2.0:1789762825.430594:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067367424 00000100:00100000:2.0:1789762825.430624:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 139 00000100:00100000:2.0:1789762825.430630:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:13930:x1876702067367424:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762825.430637:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762825.430645:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008684e667/0xdf5a7ac59f175726 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c71a expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.430671:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(2 0) 00010000:00010000:2.0:1789762825.430675:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762825.430679:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008684e667/0xdf5a7ac59f175726 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c71a expref: 20 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.430687:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008684e667/0xdf5a7ac59f175726 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c71a expref: 20 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.430694:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d10aec00, rc 0) 00000100:00100000:2.0:1789762825.430720:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:13930:x1876702067367424:12345-192.168.203.37@tcp:101:dd.0 Request processed in 91us (212us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.430725:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 139 00000100:00100000:2.0:1789762825.433898:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.433938:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067367552 00000100:00100000:0.0:1789762825.433944:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067367552 00000100:00100000:0.0:1789762825.433973:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 140 00000100:00100000:0.0:1789762825.433980:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:13937:x1876702067367552:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762825.433987:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762825.433994:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17572d lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c721 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.434008:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(3 0) 00010000:00010000:0.0:1789762825.434011:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(4 0) 00010000:00010000:0.0:1789762825.434015:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17572d lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c721 expref: 21 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.434022:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17572d lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c721 expref: 21 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.434029:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a95a00, rc 0) 00000100:00100000:0.0:1789762825.434055:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:13937:x1876702067367552:12345-192.168.203.37@tcp:101:dd.0 Request processed in 76us (159us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762825.434059:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 140 00000100:00100000:1.0:1789762825.439589:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.439676:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067367680 00000100:00100000:2.0:1789762825.439682:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067367680 00000100:00100000:2.0:1789762825.439710:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 141 00000100:00100000:2.0:1789762825.439716:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13931:x1876702067367680:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762825.439731:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762825.439741:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c728 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762825.439758:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x9:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789762825.439767:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=298s v=(0 0) 00010000:00010000:2.0:1789762825.439770:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762825.439776:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 19 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.439785:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 19 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.439792:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d10af800, rc 0) 00000100:00100000:2.0:1789762825.439819:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:13931:x1876702067367680:12345-192.168.203.37@tcp:101:dd.0 Request processed in 103us (232us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.439823:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 141 00000100:00100000:1.0:1789762825.441473:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.441548:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067367808 00000100:00100000:2.0:1789762825.441553:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067367808 00002000:00100000:2.0:1789762825.441574:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0x9:0x0] (1048576->2097151) req@ffff9c37e3bdd180 x1876702067367808/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762825.441582:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bdd180 with x1876702067367808 ext(1048576->2097151) 00010000:00010000:2.0:1789762825.441587:0:8057:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 21 pid: 8052 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.441597:0:8057:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 21 pid: 8052 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762825.441610:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9c37e3bdd180 00000100:00100000:2.0:1789762825.441622:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 263318219682 time 263359665458 state 0x0 ref 1 00000100:00100000:2.0:1789762825.441628:0:8057:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 5: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 263318219682 time 263359665458 state 0x0 ref 1 00000100:00100000:2.0:1789762825.441641:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126500 gen 0 rate 20 depth 3 token 3 deadline 298738718218 time 298688718218 state 0x1 ref 1 00000100:00100000:2.0:1789762825.441675:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126500 gen 0 rate 20 depth 3 token 2 deadline 298738718218 time 298688753072 state 0x0 ref 1 00000100:00100000:2.0:1789762825.441683:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:2398:x1876702067367808:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762825.441999:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.442050:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067367936 00000100:00100000:0.0:1789762825.442056:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067367936 00000100:00100000:0.0:1789762825.442084:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 142 00000100:00100000:0.0:1789762825.442092:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:13942:x1876702067367936:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762825.442100:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762825.442114:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17573b lrc: 2/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c72f expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.442133:0:8053:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 20 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.442141:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762825.442145:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762825.442151:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17573b lrc: 4/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c72f expref: 20 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.442177:0:8053:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 20 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.442190:0:8053:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 20 pid: 8054 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762825.442202:0:8053:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074671488:192.168.203.37@tcp:104: 00000100:00100000:0.0:1789762825.442254:0:8053:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c9a9d000 going to sleep for 11 seconds 00000100:00100000:0.0:1789762825.442951:0:8053:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074671488:192.168.203.37@tcp:104: 00010000:00010000:0.0:1789762825.442967:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17573b lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c72f expref: 20 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.442977:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a95000, rc 0) 00000100:00100000:0.0:1789762825.443006:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:13942:x1876702067367936:12345-192.168.203.37@tcp:101:dd.0 Request processed in 916us (1009us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762825.443020:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 142 00000100:00100000:0.0:1789762825.447523:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:2398:x1876702067367808:12345-192.168.203.37@tcp:4:dd.0 Request processed in 5838us (6048us total) trans 4294967368 rc 0/0 00000100:00100000:0.0:1789762825.447531:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 2 00000100:00100000:0.0:1789762825.447538:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126500 gen 0 rate 20 depth 3 token 2 deadline 298738718218 time 298688753072 state 0x2 ref 0 00002000:00100000:0.0:1789762825.447548:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0x9:0x0] (1048576->2097151) req@ffff9c37e3bdd180 x1876702067367808/t4294967368(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.447556:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bdd180 with x1876702067367808 ext(1048576->2097151) 00010000:00010000:0.0:1789762825.447562:0:8057:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 23 pid: 8052 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.447574:0:8057:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 23 pid: 8052 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762825.447583:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9c37e3bdd180 00000100:00100000:1.0:1789762825.449613:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.449666:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067368064 00000100:00100000:2.0:1789762825.449671:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067368064 00010000:00010000:2.0:1789762825.449692:0:6404:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 23 pid: 8052 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762825.449719:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 71 00000100:00100000:2.0:1789762825.449725:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:2397:x1876702067368064:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762825.449734:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1789762825.449743:0:6404:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0x9:0x0] updating lvb size from disk: 0 -> 2097152 00002000:00010000:2.0:1789762825.449746:0:6404:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0x9:0x0] updating lvb mtime from disk: 0 -> 1789762825 00002000:00010000:2.0:1789762825.449748:0:6404:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0x9:0x0] updating lvb ctime from disk: 0 -> 1789762825 00010000:00010000:2.0:1789762825.449752:0:6404:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 23 pid: 8052 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.449767:0:6404:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 23 pid: 8052 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.449783:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af6a0ed4/0xdf5a7ac59f175711 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c705 expref: 23 pid: 8052 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.449793:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762825.449796:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762825.449801:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175718 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [0->2097151] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c70c expref: 22 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.449808:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175718 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [0->2097151] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c70c expref: 22 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.449813:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762825.449815:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762825.449818:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000cd624d43/0xdf5a7ac59f17571f lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c713 expref: 22 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.449824:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cd624d43/0xdf5a7ac59f17571f lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c713 expref: 22 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.449828:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762825.449829:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762825.449832:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008684e667/0xdf5a7ac59f175726 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [3145728->6291455] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c71a expref: 22 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.449842:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008684e667/0xdf5a7ac59f175726 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [3145728->6291455] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c71a expref: 22 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.449847:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762825.449848:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762825.449851:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17572d lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [6291456->18446744073709551615] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c721 expref: 22 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.449856:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17572d lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [6291456->18446744073709551615] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c721 expref: 22 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.449876:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17572d lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [6291456->18446744073709551615] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c721 expref: 22 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762825.449884:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074671616:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762825.449955:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008684e667/0xdf5a7ac59f175726 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [3145728->6291455] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c71a expref: 22 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762825.449963:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074671744:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762825.449992:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000cd624d43/0xdf5a7ac59f17571f lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c713 expref: 22 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762825.449999:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074671872:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762825.450034:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175718 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [0->2097151] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c70c expref: 22 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762825.450043:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074672000:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762825.450062:0:6404:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37e12bdcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762825.451260:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074671616:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762825.451277:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074671744:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762825.451290:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074671872:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762825.451298:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074672000:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762825.451317:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762825.451346:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:2397:x1876702067368064:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 1620us (1734us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.451354:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 71 00000100:00100000:1.0:1789762825.454379:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.454454:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067368192 00000100:00100000:0.0:1789762825.454459:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067368192 00000100:00100000:0.0:1789762825.454488:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 143 00000100:00100000:0.0:1789762825.454494:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:13932:x1876702067368192:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762825.454505:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762825.454515:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175742 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c736 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.454530:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762825.454539:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(2 0) 00010000:00010000:0.0:1789762825.454543:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175742 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c736 expref: 21 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.454553:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175742 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c736 expref: 21 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.454560:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94400, rc 0) 00000100:00100000:0.0:1789762825.454587:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:13932:x1876702067368192:12345-192.168.203.37@tcp:101:dd.0 Request processed in 93us (209us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762825.454592:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 143 00000100:00100000:1.0:1789762825.461622:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.461704:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067368320 00000100:00100000:2.0:1789762825.461709:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067368320 00000100:00100000:2.0:1789762825.461737:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 144 00000100:00100000:2.0:1789762825.461743:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:13941:x1876702067368320:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762825.461751:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762825.461758:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e30461e/0xdf5a7ac59f175749 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c73d expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.461774:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x9:0x0].0x0 since=0s v=(2 0) 00010000:00010000:2.0:1789762825.461777:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762825.461781:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e30461e/0xdf5a7ac59f175749 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 5 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c73d expref: 22 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.461795:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e30461e/0xdf5a7ac59f175749 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 5 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c73d expref: 22 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.461803:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d10ae600, rc 0) 00000100:00100000:2.0:1789762825.461829:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:13941:x1876702067368320:12345-192.168.203.37@tcp:101:dd.0 Request processed in 87us (210us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.461834:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 144 00000100:00100000:1.0:1789762825.467487:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.467561:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067368448 00000100:00100000:0.0:1789762825.467567:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067368448 00000100:00100000:0.0:1789762825.467596:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 145 00000100:00100000:0.0:1789762825.467602:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:13939:x1876702067368448:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762825.467609:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762825.467616:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175750 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c744 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.467631:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x9:0x0].0x0 since=0s v=(3 0) 00010000:00010000:0.0:1789762825.467634:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(4 0) 00010000:00010000:0.0:1789762825.467638:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175750 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 6 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c744 expref: 23 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.467645:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175750 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 6 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c744 expref: 23 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.467681:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94800, rc 0) 00000100:00100000:0.0:1789762825.467709:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:13939:x1876702067368448:12345-192.168.203.37@tcp:101:dd.0 Request processed in 107us (224us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762825.467714:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 145 00000100:00100000:1.0:1789762825.470487:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.470561:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067368576 00000100:00100000:0.0:1789762825.470567:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067368576 00000100:00100000:0.0:1789762825.470594:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 146 00000100:00100000:0.0:1789762825.470601:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:13940:x1876702067368576:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762825.470608:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762825.470615:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175757 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c74b expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.470629:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x9:0x0].0x0 since=0s v=(4 0) 00010000:00010000:0.0:1789762825.470632:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(5 0) 00010000:00010000:0.0:1789762825.470636:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175757 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 7 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c74b expref: 24 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.470644:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175757 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 7 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c74b expref: 24 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.470650:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a94000, rc 0) 00000100:00100000:0.0:1789762825.470694:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:13940:x1876702067368576:12345-192.168.203.37@tcp:101:dd.0 Request processed in 93us (208us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762825.470699:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 146 00000100:00100000:1.0:1789762825.471444:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.471519:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067368704 00000100:00100000:2.0:1789762825.471524:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067368704 00000100:00100000:2.0:1789762825.471551:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 147 00000100:00100000:2.0:1789762825.471557:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:13933:x1876702067368704:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762825.471564:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762825.471572:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ed78738/0xdf5a7ac59f17575e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c752 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.471585:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x9:0x0].0x0 since=0s v=(5 0) 00010000:00010000:2.0:1789762825.471589:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(6 0) 00010000:00010000:2.0:1789762825.471593:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed78738/0xdf5a7ac59f17575e lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 8 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c752 expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.471601:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006ed78738/0xdf5a7ac59f17575e lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 8 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c752 expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762825.471608:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d10afe00, rc 0) 00000100:00100000:2.0:1789762825.471634:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:13933:x1876702067368704:12345-192.168.203.37@tcp:101:dd.0 Request processed in 77us (192us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.471645:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 147 00000100:00100000:1.0:1789762825.473964:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.474011:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067368832 00000100:00100000:0.0:1789762825.474016:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067368832 00000100:00100000:0.0:1789762825.474043:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 148 00000100:00100000:0.0:1789762825.474049:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:13934:x1876702067368832:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762825.474056:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762825.474063:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175765 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c759 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.474076:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x9:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762825.474080:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762825.474084:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175765 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c759 expref: 26 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.474092:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175765 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c759 expref: 26 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762825.474099:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a95c00, rc 0) 00000100:00100000:0.0:1789762825.474124:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:13934:x1876702067368832:12345-192.168.203.37@tcp:101:dd.0 Request processed in 75us (162us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762825.474128:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 148 00000100:00100000:0.0:1789762825.474826:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298771896207 time 298721896207 state 0x1 ref 8 00000100:00100000:0.0:1789762825.474839:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2398:x1876702067365504:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762825.478623:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2398:x1876702067365504:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3781us (168361us total) trans 4294967679 rc 0/0 00000100:00100000:0.0:1789762825.478634:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 239 00002000:00100000:0.0:1789762825.478650:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x8:0x0] (4194304->5242879) req@ffff9c37c94a2300 x1876702067365504/t4294967679(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.478673:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c94a2300 with x1876702067365504 ext(4194304->5242879) 00002000:00010000:0.0:1789762825.478678:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c94a2300 00000100:00100000:1.0:1789762825.486089:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762825.486146:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067368960 00000100:00100000:2.0:1789762825.486155:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067368960 00002000:00100000:2.0:1789762825.486190:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0x9:0x0] (6291456->7340031) req@ffff9c37ee763480 x1876702067368960/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762825.486199:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee763480 with x1876702067368960 ext(6291456->7340031) 00002000:00010000:2.0:1789762825.486203:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ee763480 00000100:00100000:2.0:1789762825.486218:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298771896207 time 298721896207 state 0x1 ref 8 00000100:00100000:2.0:1789762825.497333:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762825.497425:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067369088 00000100:00100000:1.0:1789762825.497434:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067369088 00002000:00100000:1.0:1789762825.497470:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0x9:0x0] (2097152->3145727) req@ffff9c37d184ca80 x1876702067369088/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762825.497480:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184ca80 with x1876702067369088 ext(2097152->3145727) 00002000:00010000:1.0:1789762825.497484:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184ca80 00000100:00100000:1.0:1789762825.497497:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298771896207 time 298721896207 state 0x1 ref 9 00000100:00100000:2.0:1789762825.498589:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.498623:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067369216 00000100:00100000:3.0:1789762825.498629:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067369216 00002000:00100000:3.0:1789762825.498650:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x9:0x0] (0->1048575) req@ffff9c37d184ed80 x1876702067369216/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.498670:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184ed80 with x1876702067369216 ext(0->1048575) 00010000:00010000:3.0:1789762825.498688:0:8058:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 27 pid: 8054 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.498702:0:8058:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 27 pid: 8054 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762825.498708:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9c37d184ed80 00000100:00100000:3.0:1789762825.498718:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126500 gen 0 rate 20 depth 3 token 2 deadline 298738718218 time 298688753072 state 0x0 ref 1 00000100:00100000:3.0:1789762825.498726:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126500 gen 0 rate 20 depth 3 token 2 deadline 298738753072 time 298688753072 state 0x1 ref 1 00000100:00100000:3.0:1789762825.498745:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126500 gen 0 rate 20 depth 3 token 2 deadline 298738753072 time 298745823862 state 0x0 ref 1 00000100:00100000:3.0:1789762825.498755:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2397:x1876702067369216:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762825.504016:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:2397:x1876702067369216:12345-192.168.203.37@tcp:4:dd.0 Request processed in 5259us (5432us total) trans 4294967680 rc 0/0 00000100:00100000:3.0:1789762825.504028:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 3 00000100:00100000:3.0:1789762825.504038:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126500 gen 0 rate 20 depth 3 token 2 deadline 298738753072 time 298745823862 state 0x2 ref 0 00002000:00100000:3.0:1789762825.504050:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x9:0x0] (0->1048575) req@ffff9c37d184ed80 x1876702067369216/t4294967680(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.504060:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184ed80 with x1876702067369216 ext(0->1048575) 00010000:00010000:3.0:1789762825.504068:0:8058:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 29 pid: 8054 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762825.504094:0:8058:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 29 pid: 8054 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762825.504102:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9c37d184ed80 00000100:00100000:2.0:1789762825.505958:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762825.505998:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067369344 00000100:00100000:1.0:1789762825.506004:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067369344 00010000:00010000:1.0:1789762825.506024:0:6406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 29 pid: 8054 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762825.506050:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 72 00000100:00100000:1.0:1789762825.506062:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:2398:x1876702067369344:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762825.506072:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1789762825.506084:0:6406:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x9:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:1.0:1789762825.506087:0:6406:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x9:0x0] updating lvb mtime from disk: 0 -> 1789762825 00002000:00010000:1.0:1789762825.506090:0:6406:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x9:0x0] updating lvb ctime from disk: 0 -> 1789762825 00010000:00010000:1.0:1789762825.506096:0:6406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 29 pid: 8054 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506110:0:6406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 29 pid: 8054 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506133:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b19d5689/0xdf5a7ac59f175734 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c728 expref: 29 pid: 8054 timeout: 398 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506148:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506151:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506159:0:6406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17573b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c72f expref: 28 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506166:0:6406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17573b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c72f expref: 28 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506173:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506174:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506178:0:6406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175742 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c736 expref: 28 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506194:0:6406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175742 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c736 expref: 28 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506199:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506201:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506204:0:6406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005e30461e/0xdf5a7ac59f175749 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c73d expref: 28 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506209:0:6406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e30461e/0xdf5a7ac59f175749 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c73d expref: 28 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506214:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506216:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506219:0:6406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175750 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c744 expref: 28 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506224:0:6406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175750 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c744 expref: 28 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506228:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506229:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506232:0:6406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175757 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c74b expref: 28 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506237:0:6406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175757 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c74b expref: 28 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506246:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506248:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506251:0:6406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006ed78738/0xdf5a7ac59f17575e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c752 expref: 28 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506256:0:6406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ed78738/0xdf5a7ac59f17575e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c752 expref: 28 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506260:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506262:0:6406:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762825.506264:0:6406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175765 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c759 expref: 28 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506269:0:6406:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175765 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c759 expref: 28 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762825.506303:0:6406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175765 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c759 expref: 28 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762825.506313:0:6406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee761f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672128:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762825.506369:0:6406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000006ed78738/0xdf5a7ac59f17575e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c752 expref: 28 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762825.506377:0:6406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee763b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672256:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762825.506410:0:6406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175757 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c74b expref: 28 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762825.506418:0:6406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee763100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672384:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762825.506445:0:6406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175750 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c744 expref: 28 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762825.506453:0:6406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee761500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672512:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762825.506479:0:6406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005e30461e/0xdf5a7ac59f175749 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c73d expref: 28 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762825.506486:0:6406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee761880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672640:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762825.506512:0:6406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175742 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c736 expref: 28 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762825.506521:0:6406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee762680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672768:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762825.506545:0:6406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17573b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c72f expref: 28 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762825.506553:0:6406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee762a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672896:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762825.506572:0:6406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7b7e6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762825.510489:0:6406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee761f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672128:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762825.510496:0:6406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee763b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672256:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762825.510510:0:6406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee763100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672384:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762825.510515:0:6406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee761500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672512:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762825.510520:0:6406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee761880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672640:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762825.510525:0:6406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee762680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672768:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762825.510529:0:6406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee762a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6406:1876702074672896:192.168.203.37@tcp:105: 00010000:00010000:1.0:1789762825.510548:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762825.510577:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2398:x1876702067369344:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 4515us (4620us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762825.510584:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 72 00000100:00100000:0.0:1789762825.524870:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298821942405 time 298771942405 state 0x1 ref 9 00000100:00100000:0.0:1789762825.524885:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2398:x1876702067365632:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762825.527581:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:2398:x1876702067365632:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2699us (210981us total) trans 4294967681 rc 0/0 00000100:00100000:0.0:1789762825.527588:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 240 00002000:00100000:0.0:1789762825.527595:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x8:0x0] (5242880->6291455) req@ffff9c37e3bde680 x1876702067365632/t4294967681(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.527602:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bde680 with x1876702067365632 ext(5242880->6291455) 00002000:00010000:0.0:1789762825.527606:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e3bde680 00000100:00100000:2.0:1789762825.540363:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.540407:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067369472 00000100:00100000:0.0:1789762825.540414:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067369472 00002000:00100000:0.0:1789762825.540439:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0x9:0x0] (0->1048575) req@ffff9c37d184d180 x1876702067369472/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.540447:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184d180 with x1876702067369472 ext(0->1048575) 00002000:00010000:0.0:1789762825.540450:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184d180 00000100:00100000:0.0:1789762825.540463:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298821942405 time 298771942405 state 0x1 ref 9 00000100:00100000:1.0:1789762825.574962:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298872031214 time 298822031214 state 0x1 ref 9 00000100:00100000:1.0:1789762825.574979:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2398:x1876702067365760:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762825.575965:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.576026:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067369600 00000100:00100000:0.0:1789762825.576035:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067369600 00002000:00100000:0.0:1789762825.576072:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x9:0x0] (1048576->2097151) req@ffff9c37d184d500 x1876702067369600/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.576095:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184d500 with x1876702067369600 ext(1048576->2097151) 00002000:00010000:0.0:1789762825.576101:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184d500 00000100:00100000:0.0:1789762825.576115:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298872031214 time 298822031214 state 0x1 ref 10 00000100:00100000:3.0:1789762825.577701:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2398:x1876702067365760:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2725us (254251us total) trans 4294967682 rc 0/0 00000100:00100000:3.0:1789762825.577708:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 241 00002000:00100000:3.0:1789762825.577715:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0x8:0x0] (6291456->7340031) req@ffff9c37d8c9c700 x1876702067365760/t4294967682(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.577722:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c9c700 with x1876702067365760 ext(6291456->7340031) 00002000:00010000:3.0:1789762825.577726:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8c9c700 00000100:00100000:2.0:1789762825.579883:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.579934:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067369728 00000100:00100000:3.0:1789762825.579939:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067369728 00002000:00100000:3.0:1789762825.579958:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x9:0x0] (4194304->5242879) req@ffff9c37d184dc00 x1876702067369728/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.579969:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184dc00 with x1876702067369728 ext(4194304->5242879) 00002000:00010000:3.0:1789762825.579973:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184dc00 00000100:00100000:3.0:1789762825.579986:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298872031214 time 298822031214 state 0x1 ref 10 00000100:00100000:1.0:1789762825.582972:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.583024:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067369856 00000100:00100000:0.0:1789762825.583028:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067369856 00002000:00100000:0.0:1789762825.583046:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x9:0x0] (5242880->6291455) req@ffff9c37ee762a00 x1876702067369856/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.583054:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee762a00 with x1876702067369856 ext(5242880->6291455) 00002000:00010000:0.0:1789762825.583058:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ee762a00 00000100:00100000:0.0:1789762825.583066:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298872031214 time 298822031214 state 0x1 ref 11 00000100:00100000:2.0:1789762825.585024:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.585066:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067369984 00000100:00100000:0.0:1789762825.585072:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067369984 00002000:00100000:0.0:1789762825.585091:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x9:0x0] (2097152->3145727) req@ffff9c37d184e300 x1876702067369984/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.585098:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184e300 with x1876702067369984 ext(2097152->3145727) 00002000:00010000:0.0:1789762825.585102:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184e300 00000100:00100000:0.0:1789762825.585110:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298872031214 time 298822031214 state 0x1 ref 12 00000100:00100000:2.0:1789762825.588422:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.588458:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067370112 00000100:00100000:3.0:1789762825.588464:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067370112 00002000:00100000:3.0:1789762825.588485:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0x9:0x0] (3145728->4194303) req@ffff9c37d184c000 x1876702067370112/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.588509:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184c000 with x1876702067370112 ext(3145728->4194303) 00002000:00010000:3.0:1789762825.588512:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184c000 00000100:00100000:3.0:1789762825.588521:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298872031214 time 298822031214 state 0x1 ref 13 00000100:00100000:1.0:1789762825.590686:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762825.590717:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.590738:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067370240 00000100:00100000:0.0:1789762825.590743:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067370240 00002000:00100000:0.0:1789762825.590767:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x9:0x0] (7340032->8388607) req@ffff9c37ee762680 x1876702067370240/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1789762825.590770:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067370368 00000100:00100000:2.0:1789762825.590777:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067370368 00002000:00100000:0.0:1789762825.590790:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee762680 with x1876702067370240 ext(7340032->8388607) 00002000:00010000:0.0:1789762825.590795:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ee762680 00002000:00100000:2.0:1789762825.590803:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x9:0x0] (6291456->7340031) req@ffff9c37ee761880 x1876702067370368/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1789762825.590812:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298872031214 time 298822031214 state 0x1 ref 14 00002000:00100000:2.0:1789762825.590826:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee761880 with x1876702067370368 ext(6291456->7340031) 00002000:00010000:2.0:1789762825.590830:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ee761880 00000100:00100000:2.0:1789762825.590842:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298872031214 time 298822031214 state 0x1 ref 15 00000100:00100000:2.0:1789762825.625037:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298922108915 time 298872108915 state 0x1 ref 15 00000100:00100000:2.0:1789762825.625054:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2398:x1876702067365888:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762825.627470:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2398:x1876702067365888:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2419us (295892us total) trans 4294967683 rc 0/0 00000100:00100000:0.0:1789762825.627477:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 242 00002000:00100000:0.0:1789762825.627485:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0x8:0x0] (7340032->8388607) req@ffff9c37e3bdc000 x1876702067365888/t4294967683(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.627493:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bdc000 with x1876702067365888 ext(7340032->8388607) 00002000:00010000:0.0:1789762825.627497:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e3bdc000 00000100:00100000:3.0:1789762825.675127:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298972191549 time 298922191549 state 0x1 ref 14 00000100:00100000:3.0:1789762825.675154:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:2398:x1876702067366016:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762825.681719:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2398:x1876702067366016:12345-192.168.203.37@tcp:4:dd.0 Request processed in 6573us (343766us total) trans 4294967369 rc 0/0 00000100:00100000:3.0:1789762825.681728:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 243 00002000:00100000:3.0:1789762825.681739:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0x8:0x0] (4194304->5242879) req@ffff9c37d8c9f800 x1876702067366016/t4294967369(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.681750:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c9f800 with x1876702067366016 ext(4194304->5242879) 00002000:00010000:3.0:1789762825.681756:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8c9f800 00000100:00100000:2.0:1789762825.690243:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.690297:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067370496 00000100:00100000:3.0:1789762825.690305:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067370496 00002000:00100000:3.0:1789762825.690339:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x9:0x0] (3145728->4194303) req@ffff9c37d184c380 x1876702067370496/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.690351:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184c380 with x1876702067370496 ext(3145728->4194303) 00002000:00010000:3.0:1789762825.690355:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184c380 00000100:00100000:3.0:1789762825.690372:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298972191549 time 298922191549 state 0x1 ref 14 00000100:00100000:1.0:1789762825.690899:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.690954:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067370624 00000100:00100000:3.0:1789762825.690966:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067370624 00002000:00100000:3.0:1789762825.690987:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x9:0x0] (7340032->8388607) req@ffff9c37ee761500 x1876702067370624/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.690998:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee761500 with x1876702067370624 ext(7340032->8388607) 00002000:00010000:3.0:1789762825.691002:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ee761500 00000100:00100000:3.0:1789762825.691013:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 298972191549 time 298922191549 state 0x1 ref 15 00000100:00100000:0.0:1789762825.725207:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299022273338 time 298972273338 state 0x1 ref 15 00000100:00100000:0.0:1789762825.725228:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2398:x1876702067366144:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762825.728066:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074673024:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762825.728117:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762825.728164:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074673024 00000100:00100000:0.0:1789762825.728172:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:2398:x1876702067366144:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2947us (384068us total) trans 4294967370 rc 0/0 00000100:00100000:2.0:1789762825.728172:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074673024 00000100:00100000:0.0:1789762825.728186:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 244 00002000:00100000:0.0:1789762825.728198:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x8:0x0] (5242880->6291455) req@ffff9c37e3bdfb80 x1876702067366144/t4294967370(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.728209:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bdfb80 with x1876702067366144 ext(5242880->6291455) 00000100:00100000:2.0:1789762825.728212:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 270 00002000:00010000:0.0:1789762825.728218:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3bdfb80 00000100:00100000:2.0:1789762825.728225:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074673024:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762825.728254:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762825.728292:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074673024:12345-0@lo:41:osp-pre-1-0.0 Request processed in 72us (177us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762825.728297:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 270 00000100:00100000:3.0:1789762825.728357:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074673024:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762825.730707:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.730755:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067370880 00000100:00100000:0.0:1789762825.730766:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067370880 00000100:00100000:0.0:1789762825.730797:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 56 00000100:00100000:0.0:1789762825.730803:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:13936:x1876702067370880:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:0.0:1789762825.731051:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:13936:x1876702067370880:12345-192.168.203.37@tcp:35:dd.0 Request processed in 246us (344us total) trans 4294967387 rc 0/0 00000100:00100000:0.0:1789762825.731060:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 56 00000100:00100000:1.0:1789762825.732448:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762825.732496:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067371008 00000100:00100000:0.0:1789762825.732502:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067371008 00002000:00100000:0.0:1789762825.732526:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0x9:0x0] (4194304->5242879) req@ffff9c37ee763100 x1876702067371008/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.732534:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee763100 with x1876702067371008 ext(4194304->5242879) 00002000:00010000:0.0:1789762825.732537:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ee763100 00000100:00100000:0.0:1789762825.732548:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299022273338 time 298972273338 state 0x1 ref 15 00000100:00100000:2.0:1789762825.733473:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762825.733513:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067371136 00000100:00100000:3.0:1789762825.733519:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067371136 00002000:00100000:3.0:1789762825.733540:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x9:0x0] (5242880->6291455) req@ffff9c37d184df80 x1876702067371136/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/0 e 0 to 0 dl 1789762836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.733550:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184df80 with x1876702067371136 ext(5242880->6291455) 00002000:00010000:3.0:1789762825.733554:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184df80 00000100:00100000:3.0:1789762825.733565:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299022273338 time 298972273338 state 0x1 ref 16 00000100:00100000:3.0:1789762825.775283:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299072350424 time 299022350424 state 0x1 ref 16 00000100:00100000:3.0:1789762825.775305:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:2397:x1876702067366272:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762825.778149:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2397:x1876702067366272:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2850us (398706us total) trans 4294967371 rc 0/0 00000100:00100000:3.0:1789762825.778159:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 245 00002000:00100000:3.0:1789762825.778178:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x8:0x0] (6291456->7340031) req@ffff9c37e3bdf480 x1876702067366272/t4294967371(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.778189:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e3bdf480 with x1876702067366272 ext(6291456->7340031) 00002000:00010000:3.0:1789762825.778194:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37e3bdf480 00000100:00100000:2.0:1789762825.825371:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299122439591 time 299072439591 state 0x1 ref 15 00000100:00100000:2.0:1789762825.825392:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2398:x1876702067366400:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762825.828211:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:2398:x1876702067366400:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2825us (442493us total) trans 4294967372 rc 0/0 00000100:00100000:0.0:1789762825.828219:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 246 00002000:00100000:0.0:1789762825.828228:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0x8:0x0] (7340032->8388607) req@ffff9c37f6089c00 x1876702067366400/t4294967372(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.828236:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f6089c00 with x1876702067366400 ext(7340032->8388607) 00002000:00010000:0.0:1789762825.828240:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f6089c00 00010000:00010000:0.0:1789762825.856047:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762825.875441:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299172506576 time 299122506576 state 0x1 ref 14 00000100:00100000:0.0:1789762825.875466:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2398:x1876702067368960:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762825.878533:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2398:x1876702067368960:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3075us (392450us total) trans 4294967373 rc 0/0 00000100:00100000:0.0:1789762825.878541:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 247 00002000:00100000:0.0:1789762825.878550:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0x9:0x0] (6291456->7340031) req@ffff9c37ee763480 x1876702067368960/t4294967373(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.878558:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee763480 with x1876702067368960 ext(6291456->7340031) 00002000:00010000:0.0:1789762825.878564:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ee763480 00000100:00100000:0.0:1789762825.925499:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299222564326 time 299172564326 state 0x1 ref 13 00000100:00100000:0.0:1789762825.925520:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:2397:x1876702067369088:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762825.928485:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2397:x1876702067369088:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2971us (431155us total) trans 4294967374 rc 0/0 00000100:00100000:0.0:1789762825.928493:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 248 00002000:00100000:0.0:1789762825.928502:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0x9:0x0] (2097152->3145727) req@ffff9c37d184ca80 x1876702067369088/t4294967374(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762825.928509:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184ca80 with x1876702067369088 ext(2097152->3145727) 00002000:00010000:0.0:1789762825.928514:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184ca80 00000100:00100000:3.0:1789762825.975611:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299272672939 time 299222672939 state 0x1 ref 12 00000100:00100000:3.0:1789762825.975639:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2398:x1876702067369472:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762825.981953:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2398:x1876702067369472:12345-192.168.203.37@tcp:4:dd.0 Request processed in 6323us (441589us total) trans 4294967375 rc 0/0 00000100:00100000:3.0:1789762825.981962:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 249 00002000:00100000:3.0:1789762825.981972:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0x9:0x0] (0->1048575) req@ffff9c37d184d180 x1876702067369472/t4294967375(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762825.981980:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184d180 with x1876702067369472 ext(0->1048575) 00002000:00010000:3.0:1789762825.981985:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184d180 00000100:00100000:3.0:1789762826.025682:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299322749042 time 299272749042 state 0x1 ref 11 00000100:00100000:3.0:1789762826.025706:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2395:x1876702067369600:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762826.028530:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2395:x1876702067369600:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2832us (452564us total) trans 4294967684 rc 0/0 00000100:00100000:3.0:1789762826.028537:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 250 00002000:00100000:3.0:1789762826.028547:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0x9:0x0] (1048576->2097151) req@ffff9c37d184d500 x1876702067369600/t4294967684(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762826.028555:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184d500 with x1876702067369600 ext(1048576->2097151) 00002000:00010000:3.0:1789762826.028568:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184d500 00000100:00100000:0.0:1789762826.075810:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299372876586 time 299322876586 state 0x1 ref 10 00000100:00100000:0.0:1789762826.075835:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2397:x1876702067369728:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762826.078574:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2397:x1876702067369728:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2749us (498691us total) trans 4294967685 rc 0/0 00000100:00100000:0.0:1789762826.078590:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 251 00002000:00100000:0.0:1789762826.078599:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0x9:0x0] (4194304->5242879) req@ffff9c37d184dc00 x1876702067369728/t4294967685(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762826.078607:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184dc00 with x1876702067369728 ext(4194304->5242879) 00002000:00010000:0.0:1789762826.078612:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184dc00 00000100:00100000:3.0:1789762826.125971:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299423032538 time 299373032538 state 0x1 ref 9 00000100:00100000:3.0:1789762826.126002:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee762a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2396:x1876702067369856:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762826.129990:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee762a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2396:x1876702067369856:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3998us (547014us total) trans 4294967686 rc 0/0 00000100:00100000:3.0:1789762826.130000:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 252 00002000:00100000:3.0:1789762826.130012:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0x9:0x0] (5242880->6291455) req@ffff9c37ee762a00 x1876702067369856/t4294967686(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762826.130023:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee762a00 with x1876702067369856 ext(5242880->6291455) 00002000:00010000:3.0:1789762826.130029:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ee762a00 00000100:00100000:0.0:1789762826.176043:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299473108022 time 299423108022 state 0x1 ref 8 00000100:00100000:0.0:1789762826.176068:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2398:x1876702067369984:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762826.179066:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2398:x1876702067369984:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3004us (594038us total) trans 4294967687 rc 0/0 00000100:00100000:0.0:1789762826.179075:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 253 00002000:00100000:0.0:1789762826.179085:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0x9:0x0] (2097152->3145727) req@ffff9c37d184e300 x1876702067369984/t4294967687(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762826.179093:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184e300 with x1876702067369984 ext(2097152->3145727) 00002000:00010000:0.0:1789762826.179098:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184e300 00000100:00100000:0.0:1789762826.226099:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299523164463 time 299473164463 state 0x1 ref 7 00000100:00100000:0.0:1789762826.226126:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2395:x1876702067370112:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762826.228883:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:2395:x1876702067370112:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2768us (640460us total) trans 4294967688 rc 0/0 00000100:00100000:0.0:1789762826.228891:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 254 00002000:00100000:0.0:1789762826.228901:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x9:0x0] (3145728->4194303) req@ffff9c37d184c000 x1876702067370112/t4294967688(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762826.228939:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184c000 with x1876702067370112 ext(3145728->4194303) 00002000:00010000:0.0:1789762826.228945:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184c000 00000100:00100000:0.0:1789762826.276181:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299573244513 time 299523244513 state 0x1 ref 6 00000100:00100000:0.0:1789762826.276206:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee762680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2396:x1876702067370240:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762826.278935:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee762680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+44:2396:x1876702067370240:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2719us (688228us total) trans 4294967689 rc 0/0 00000100:00100000:0.0:1789762826.278944:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 255 00002000:00100000:0.0:1789762826.278954:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0x9:0x0] (7340032->8388607) req@ffff9c37ee762680 x1876702067370240/t4294967689(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762826.278962:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee762680 with x1876702067370240 ext(7340032->8388607) 00002000:00010000:0.0:1789762826.278968:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ee762680 00000100:00100000:3.0:1789762826.326280:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299623348350 time 299573348350 state 0x1 ref 5 00000100:00100000:3.0:1789762826.326299:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:2396:x1876702067370368:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762826.329210:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+45:2396:x1876702067370368:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2915us (738491us total) trans 4294967690 rc 0/0 00000100:00100000:3.0:1789762826.329218:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 256 00002000:00100000:3.0:1789762826.329226:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0x9:0x0] (6291456->7340031) req@ffff9c37ee761880 x1876702067370368/t4294967690(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762826.329234:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee761880 with x1876702067370368 ext(6291456->7340031) 00002000:00010000:3.0:1789762826.329240:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ee761880 00000100:00100000:1.0:1789762826.334750:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762826.334792:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067371264 00000100:00100000:3.0:1789762826.334799:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067371264 00000100:00100000:3.0:1789762826.334847:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 35 00000100:00100000:3.0:1789762826.334855:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+45:6823:x1876702067371264:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762826.334865:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 8 locks, starting at 0 00002000:00010000:3.0:1789762826.334878:0:9878:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x8:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:3.0:1789762826.334880:0:9878:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x8:0x0] updating lvb mtime from disk: 0 -> 1789762825 00002000:00010000:3.0:1789762826.334882:0:9878:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x8:0x0] updating lvb ctime from disk: 0 -> 1789762825 00010000:00010000:3.0:1789762826.334900:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002a8b8c9b/0xdf5a7ac59f1756d9 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6f0 expref: 45 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762826.334952:0:9878:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x9:0x0] updating lvb size from disk: 1048576 -> 8388608 00010000:00010000:3.0:1789762826.334960:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e30461e/0xdf5a7ac59f175749 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c73d expref: 44 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.334970:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000258fc572/0xdf5a7ac59f17573b lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 8 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c72f expref: 43 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.334981:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175765 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c759 expref: 42 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.334993:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ed78738/0xdf5a7ac59f17575e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c752 expref: 41 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.335001:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175757 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 5 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c74b expref: 40 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.335009:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175750 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 4 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c744 expref: 39 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.335024:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175742 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c736 expref: 38 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.335033:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762826.335059:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:6823:x1876702067371264:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 206us (311us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762826.335066:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 35 00000100:00100000:3.0:1789762826.376473:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299673536934 time 299623536934 state 0x1 ref 4 00000100:00100000:3.0:1789762826.376495:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2395:x1876702067370496:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762826.382634:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2395:x1876702067370496:12345-192.168.203.37@tcp:4:dd.0 Request processed in 6143us (692390us total) trans 4294967376 rc 0/0 00000100:00100000:3.0:1789762826.382643:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 257 00002000:00100000:3.0:1789762826.382666:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0x9:0x0] (3145728->4194303) req@ffff9c37d184c380 x1876702067370496/t4294967376(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762826.382675:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184c380 with x1876702067370496 ext(3145728->4194303) 00002000:00010000:3.0:1789762826.382680:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184c380 00000100:00100000:0.0:1789762826.426598:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299723662522 time 299673662522 state 0x1 ref 3 00000100:00100000:0.0:1789762826.426630:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2396:x1876702067370624:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762826.429738:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2396:x1876702067370624:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3121us (738837us total) trans 4294967377 rc 0/0 00000100:00100000:0.0:1789762826.429746:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 258 00002000:00100000:0.0:1789762826.429756:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0x9:0x0] (7340032->8388607) req@ffff9c37ee761500 x1876702067370624/t4294967377(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762826.429764:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee761500 with x1876702067370624 ext(7340032->8388607) 00002000:00010000:0.0:1789762826.429776:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ee761500 00000100:00100000:3.0:1789762826.476692:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299773757951 time 299723757951 state 0x1 ref 2 00000100:00100000:3.0:1789762826.476719:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2397:x1876702067371008:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762826.479793:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2397:x1876702067371008:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3083us (747343us total) trans 4294967378 rc 0/0 00000100:00100000:3.0:1789762826.479810:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 259 00002000:00100000:3.0:1789762826.479820:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0x9:0x0] (4194304->5242879) req@ffff9c37ee763100 x1876702067371008/t4294967378(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762826.479828:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee763100 with x1876702067371008 ext(4194304->5242879) 00002000:00010000:3.0:1789762826.479834:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ee763100 00000100:00100000:0.0:1789762826.526772:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299773757951 time 299773836767 state 0x0 ref 1 00000100:00100000:0.0:1789762826.526796:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2398:x1876702067371136:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762826.529867:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2398:x1876702067371136:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3079us (796393us total) trans 4294967379 rc 0/0 00000100:00100000:0.0:1789762826.529875:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 260 00000100:00100000:0.0:1789762826.529883:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299773757951 time 299773836767 state 0x2 ref 0 00002000:00100000:0.0:1789762826.529894:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0x9:0x0] (5242880->6291455) req@ffff9c37d184df80 x1876702067371136/t4294967379(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:606/0 lens 488/448 e 0 to 0 dl 1789762836 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762826.529902:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184df80 with x1876702067371136 ext(5242880->6291455) 00002000:00010000:0.0:1789762826.529908:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184df80 00000100:00100000:2.0:1789762826.533690:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762826.533733:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067371392 00000100:00100000:3.0:1789762826.533741:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067371392 00000100:00100000:3.0:1789762826.533780:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 36 00000100:00100000:3.0:1789762826.533787:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:6358:x1876702067371392:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762826.533796:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 5 locks, starting at 0 00002000:00010000:3.0:1789762826.533808:0:6405:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0x8:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:3.0:1789762826.533811:0:6405:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0x8:0x0] updating lvb mtime from disk: 0 -> 1789762825 00002000:00010000:3.0:1789762826.533813:0:6405:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0x8:0x0] updating lvb ctime from disk: 0 -> 1789762825 00002000:00010000:3.0:1789762826.533815:0:6405:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000400:0x8:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:3.0:1789762826.533832:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ee1ee3ef/0xdf5a7ac59f1756e7 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6f7 expref: 40 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762826.533854:0:6405:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0x9:0x0] updating lvb size from disk: 2097152 -> 8388608 00010000:00010000:3.0:1789762826.533859:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008684e667/0xdf5a7ac59f175726 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 6 type: EXT [3145728->6291455] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c71a expref: 39 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.533868:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17572d lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 5 type: EXT [6291456->18446744073709551615] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c721 expref: 38 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.533876:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175718 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 4 type: EXT [0->2097151] (req 0->1048575) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c70c expref: 37 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.533892:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cd624d43/0xdf5a7ac59f17571f lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c713 expref: 36 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.533900:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762826.533952:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:6358:x1876702067371392:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 165us (262us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762826.533960:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 36 00000100:00100000:2.0:1789762826.567322:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762826.567375:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762826.567399:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067371648 00000100:00100000:3.0:1789762826.567408:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067371648 00000100:00100000:0.0:1789762826.567432:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067371904 00000100:00100000:0.0:1789762826.567437:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067371904 00000100:00100000:3.0:1789762826.567463:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 220 00000100:00100000:0.0:1789762826.567471:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 221 00000100:00100000:3.0:1789762826.567473:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:13962:x1876702067371648:12345-192.168.203.37@tcp:101:dd.0 00000100:00100000:0.0:1789762826.567480:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:13963:x1876702067371904:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762826.567495:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762826.567496:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762826.567514:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f17576c lrc: 2/0,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c767 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.567514:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175773 lrc: 2/0,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c760 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.567605:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95200/0xdf5a7ac59f17577a lrc: 3/1,0 mode: --/PR res: [0x240000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.567605:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92c00/0xdf5a7ac59f175781 lrc: 3/1,0 mode: --/PR res: [0x240000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.567621:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95200/0xdf5a7ac59f17577a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.567642:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95200/0xdf5a7ac59f17577a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.567643:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92c00/0xdf5a7ac59f175781 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.567665:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92c00/0xdf5a7ac59f175781 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762826.567669:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xf:0x0]: returning size 16777216 00000004:00200000:3.0:1789762826.567672:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xf:0x0]: returning size 16777216 00010000:00010000:0.0:1789762826.567780:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95400/0xdf5a7ac59f17576c lrc: 0/0,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c767 expref: 30 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.567780:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175773 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c760 expref: 30 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.567800:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95200/0xdf5a7ac59f17577a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c767 expref: 29 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.567807:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92c00/0xdf5a7ac59f175781 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c760 expref: 28 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.567810:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xf:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:0.0:1789762826.567813:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762826.567816:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a95200, rc 0) 00010000:00010000:3.0:1789762826.567816:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xf:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:3.0:1789762826.567819:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762826.567821:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d92c00, rc 0) 00000100:00100000:0.0:1789762826.567863:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:13963:x1876702067371904:12345-192.168.203.37@tcp:101:dd.0 Request processed in 383us (488us total) trans 4294967389 rc 0/0 00000100:00100000:3.0:1789762826.567863:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:13962:x1876702067371648:12345-192.168.203.37@tcp:101:dd.0 Request processed in 390us (548us total) trans 4294967388 rc 0/0 00000100:00100000:0.0:1789762826.567871:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 221 00000100:00100000:3.0:1789762826.567875:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 220 00000100:00100000:1.0:1789762826.569319:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762826.569344:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762826.569374:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067372032 00000100:00100000:2.0:1789762826.569380:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067372160 00000100:00100000:3.0:1789762826.569380:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067372032 00000100:00100000:2.0:1789762826.569388:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067372160 00000100:00100000:3.0:1789762826.569424:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 37 00000100:00100000:3.0:1789762826.569436:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c90850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2397:x1876702067372032:12345-192.168.203.37@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1789762826.569443:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 38 00010000:00010000:3.0:1789762826.569452:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1789762826.569460:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2397:x1876702067372160:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762826.569472:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92c00/0xdf5a7ac59f175781 lrc: 0/0,0 mode: --/PR res: [0x240000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c760 expref: 28 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.569479:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762826.569497:0:6406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xf:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762826.569501:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a95200/0xdf5a7ac59f17577a lrc: 0/0,0 mode: --/PR res: [0x240000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c767 expref: 27 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.569548:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762826.569549:0:8070:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xf:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762826.569556:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762826.569588:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c90850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2397:x1876702067372032:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 154us (268us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762826.569595:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2397:x1876702067372160:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 139us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762826.569604:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762826.569612:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 38 00000100:00100000:1.0:1789762826.569629:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067372288 00000100:00100000:1.0:1789762826.569633:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067372288 00000100:00100000:3.0:1789762826.569677:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 37 00000100:00100000:1.0:1789762826.569692:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 149 00000100:00100000:0.0:1789762826.569700:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762826.569710:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:13963:x1876702067372288:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762826.569719:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762826.569731:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000026b0307a/0xdf5a7ac59f175796 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c76e expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762826.569751:0:9037:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x8:0x0] initial LVB size: 8388608, mtime: 0x6aad9d09, atime: 0x0, ctime: 0x6aad9d09, blocks: 0x1 00010000:00010000:1.0:1789762826.569765:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x8:0x0].0x0 since=299s v=(0 0) 00010000:00010000:1.0:1789762826.569769:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x8:0x0].0x0 since=0s v=(0 0) 00000100:00100000:3.0:1789762826.569776:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067372544 00010000:00010000:1.0:1789762826.569777:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000026b0307a/0xdf5a7ac59f175796 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c76e expref: 38 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762826.569781:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067372544 00010000:00010000:1.0:1789762826.569791:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000026b0307a/0xdf5a7ac59f175796 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c76e expref: 38 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.569799:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d9231800, rc 0) 00000100:00100000:1.0:1789762826.569829:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:13963:x1876702067372288:12345-192.168.203.37@tcp:101:dd.0 Request processed in 122us (230us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762826.569834:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 149 00000100:00100000:3.0:1789762826.569841:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 57 00000100:00100000:3.0:1789762826.569857:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:13962:x1876702067372544:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:3.0:1789762826.570009:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:13962:x1876702067372544:12345-192.168.203.37@tcp:35:dd.0 Request processed in 155us (314us total) trans 4294967390 rc 0/0 00000100:00100000:3.0:1789762826.570014:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 57 00000100:00100000:0.0:1789762826.574264:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762826.574298:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067372672 00000100:00100000:3.0:1789762826.574303:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067372672 00002000:00100000:3.0:1789762826.574325:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x8:0x0] (0->1048575) req@ffff9c37d8c8e680 x1876702067372672/t0(0) o3->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:607/0 lens 488/0 e 0 to 0 dl 1789762837 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762826.574333:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c8e680 with x1876702067372672 ext(0->1048575) 00002000:00010000:3.0:1789762826.574336:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8c8e680 00000100:00100000:3.0:1789762826.574349:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263760747948 time 263760689543 state 0x0 ref 1 00000100:00100000:3.0:1789762826.574355:0:8059:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 5: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 263760747948 time 263760689543 state 0x0 ref 1 00000100:00100000:3.0:1789762826.574368:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 3 deadline 299921444915 time 299821444915 state 0x1 ref 1 00000100:00100000:3.0:1789762826.574386:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 2 deadline 299921444915 time 299821464962 state 0x0 ref 1 00000100:00100000:3.0:1789762826.574394:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2396:x1876702067372672:12345-192.168.203.37@tcp:3:dd.0 00000100:00100000:3.0:1789762826.577544:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2396:x1876702067372672:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3149us (3280us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762826.577552:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 261 00000100:00100000:3.0:1789762826.577560:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 2 deadline 299921444915 time 299821464962 state 0x2 ref 0 00002000:00100000:3.0:1789762826.577584:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0x8:0x0] (0->1048575) req@ffff9c37d8c8e680 x1876702067372672/t0(0) o3->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:607/0 lens 488/440 e 0 to 0 dl 1789762837 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762826.577595:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c8e680 with x1876702067372672 ext(0->1048575) 00002000:00010000:3.0:1789762826.577599:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8c8e680 00000100:00100000:0.0:1789762826.613998:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762826.614066:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067372928 00000100:00100000:2.0:1789762826.614076:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067372928 00000100:00100000:2.0:1789762826.614129:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 58 00000100:00100000:2.0:1789762826.614140:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbe450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:13976:x1876702067372928:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:2.0:1789762826.614367:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbe450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:13976:x1876702067372928:12345-192.168.203.37@tcp:35:dd.0 Request processed in 228us (375us total) trans 4294967391 rc 0/0 00000100:00100000:2.0:1789762826.614375:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 58 00000100:00100000:0.0:1789762826.614423:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762826.614460:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067373184 00000100:00100000:2.0:1789762826.614465:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067373184 00000100:00100000:2.0:1789762826.614504:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 222 00000100:00100000:2.0:1789762826.614513:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:13977:x1876702067373184:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762826.614523:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762826.614540:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f17579d lrc: 2/0,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c775 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.614605:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7e00/0xdf5a7ac59f1757a4 lrc: 3/1,0 mode: --/CR res: [0x240000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.614635:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7e00/0xdf5a7ac59f1757a4 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0xf:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.614662:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7e00/0xdf5a7ac59f1757a4 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0xf:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762826.614674:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xf:0x0]: returning size 16777216 00010000:00010000:2.0:1789762826.614744:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f17579d lrc: 0/0,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c775 expref: 30 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.614760:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7e00/0xdf5a7ac59f1757a4 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0xf:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c775 expref: 29 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.614769:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xf:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762826.614772:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:2.0:1789762826.614776:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc7e00, rc 0) 00000100:00100000:2.0:1789762826.614805:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:13977:x1876702067373184:12345-192.168.203.37@tcp:101:dd.0 Request processed in 294us (385us total) trans 4294967392 rc 0/0 00000100:00100000:2.0:1789762826.614812:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 222 00000100:00100000:1.0:1789762826.626868:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762826.626939:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067373312 00000100:00100000:2.0:1789762826.626945:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067373312 00000100:00100000:2.0:1789762826.626978:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 223 00000100:00100000:2.0:1789762826.626985:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee762300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:13978:x1876702067373312:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:2.0:1789762826.626998:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762826.627010:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1757b2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c77c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.627066:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f1757b9 lrc: 3/1,0 mode: --/PR res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.627080:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f1757b9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.627091:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f1757b9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762826.627116:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe:0x0]: returning size 16384 00000004:00200000:2.0:1789762826.627120:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762826.627133:0:6419:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f1757b9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.627150:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1757b2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c77c expref: 31 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.627165:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f1757b9 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c77c expref: 30 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.627171:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:2.0:1789762826.627173:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762826.627175:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc7800, rc 0) 00000100:00100000:2.0:1789762826.627210:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee762300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:13978:x1876702067373312:12345-192.168.203.37@tcp:101:rm.0 Request processed in 225us (344us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762826.627217:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 223 00000100:00100000:0.0:1789762826.628310:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762826.628350:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067373568 00000100:00100000:3.0:1789762826.628355:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067373568 00000100:00100000:3.0:1789762826.628384:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 224 00000100:00100000:3.0:1789762826.628390:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:13978:x1876702067373568:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:3.0:1789762826.628398:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762826.628410:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d93400/0xdf5a7ac59f1757c0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c783 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762826.628457:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe:0x0]: returning size 16384 00010000:00010000:3.0:1789762826.628521:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d93400/0xdf5a7ac59f1757c0 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c783 expref: 32 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.628530:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37c72ba600) 00010000:00010000:3.0:1789762826.628532:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762826.628534:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d93400/0xdf5a7ac59f1757c0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c783 expref: 32 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.628545:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d93400, rc 0) 00000100:00100000:3.0:1789762826.628578:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:13978:x1876702067373568:12345-192.168.203.37@tcp:101:rm.0 Request processed in 187us (268us total) trans 4294967393 rc 301/301 00000100:00100000:3.0:1789762826.628585:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 224 00000100:00100000:0.0:1789762826.630247:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762826.630282:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067373696 00000100:00100000:3.0:1789762826.630286:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067373696 00000100:00100000:3.0:1789762826.630311:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 59 00000100:00100000:3.0:1789762826.630317:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:13978:x1876702067373696:12345-192.168.203.37@tcp:37:rm.0 00000100:00100000:3.0:1789762826.631935:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:13978:x1876702067373696:12345-192.168.203.37@tcp:37:rm.0 Request processed in 1617us (1687us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762826.631941:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 59 00000100:00100000:0.0:1789762826.633125:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762826.633156:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067373952 00000100:00100000:2.0:1789762826.633162:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067373952 00000100:00100000:2.0:1789762826.633193:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 60 00000100:00100000:2.0:1789762826.633200:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:13978:x1876702067373952:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:2.0:1789762826.633293:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:13978:x1876702067373952:12345-192.168.203.37@tcp:35:rm.0 Request processed in 94us (170us total) trans 4294967394 rc 0/0 00000100:00100000:2.0:1789762826.633299:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 60 00000100:00100000:0.0:1789762826.633966:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762826.633997:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067374208 00000100:00100000:3.0:1789762826.634001:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067374208 00000100:00100000:3.0:1789762826.634026:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 225 00000100:00100000:3.0:1789762826.634032:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:13978:x1876702067374208:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:3.0:1789762826.634040:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762826.634048:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f1757ce lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c78a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.634081:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92c00/0xdf5a7ac59f1757d5 lrc: 3/1,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.634089:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92c00/0xdf5a7ac59f1757d5 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0xe:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.634095:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92c00/0xdf5a7ac59f1757d5 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0xe:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762826.634100:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe:0x0]: returning size 16384 00010000:00010000:3.0:1789762826.634139:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f1757ce lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c78a expref: 35 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.634151:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92c00/0xdf5a7ac59f1757d5 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0xe:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c78a expref: 34 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.634162:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762826.634164:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1789762826.634166:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d92c00, rc 0) 00000100:00100000:3.0:1789762826.634192:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:13978:x1876702067374208:12345-192.168.203.37@tcp:101:rm.0 Request processed in 160us (227us total) trans 4294967395 rc 0/0 00000100:00100000:3.0:1789762826.634197:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 225 00000100:00100000:1.0:1789762826.635014:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762826.635043:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067374336 00000100:00100000:3.0:1789762826.635048:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067374336 00000100:00100000:3.0:1789762826.635072:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 226 00000100:00100000:3.0:1789762826.635078:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee760700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13978:x1876702067374336:12345-192.168.203.37@tcp:41:rm.0 00000004:00000024:3.0:1789762826.635110:0:13389:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7719616512 avail 7715422208 00000100:00100000:3.0:1789762826.635141:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee760700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13978:x1876702067374336:12345-192.168.203.37@tcp:41:rm.0 Request processed in 63us (129us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762826.635146:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 226 00000100:00100000:0.0:1789762826.635959:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762826.635984:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067374592 00000100:00100000:1.0:1789762826.635992:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067374592 00000100:00100000:1.0:1789762826.636031:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 227 00000100:00100000:1.0:1789762826.636038:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:13978:x1876702067374592:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:1.0:1789762826.636061:0:6418:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1789762826.636074:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94a00/0xdf5a7ac59f175703 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6fe expref: 34 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636091:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x10:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:1.0:1789762826.636101:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f1757b9 lrc: 0/0,0 mode: --/PR res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c77c expref: 33 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636108:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:1.0:1789762826.636110:0:6418:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762826.636119:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1757e3 lrc: 3/0,1 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636126:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1757e3 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636131:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1757e3 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636138:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1757ea lrc: 3/0,1 mode: --/PW res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636143:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1757ea lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636147:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1757ea lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636199:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1757f1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636205:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1757f1 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636217:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1757f1 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789762826.636827:0:6418:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].8, 64 off8704 00000004:00080000:1.0:1789762826.636835:0:6418:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 299883916708ns, next at 300883915268ns, rc = 0 00000040:00080000:1.0:1789762826.636884:0:6418:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].8, 64 off8704 00000004:00080000:1.0:1789762826.636888:0:6418:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 299883970448ns, next at 300883969636ns, rc = 0 00010000:00010000:1.0:1789762826.636955:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231200/0xdf5a7ac59f1757f8 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636965:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231200/0xdf5a7ac59f1757f8 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636984:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231200/0xdf5a7ac59f1757f8 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.636992:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231200/0xdf5a7ac59f1757f8 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637000:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231200/0xdf5a7ac59f1757f8 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637005:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231200/0xdf5a7ac59f1757f8 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637023:0:6418:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762826.637035:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1757f1 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.637035:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762826.637043:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231200/0xdf5a7ac59f1757f8 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637044:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1757f1 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637068:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1757f1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.637068:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762826.637075:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231200/0xdf5a7ac59f1757f8 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637077:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762826.637083:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1757f1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.637086:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231200/0xdf5a7ac59f1757f8 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637090:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1757f1 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.637097:0:6407:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x10:0x0].0x0 (ffff9c37db743400) 00010000:00010000:3.0:1789762826.637106:0:6407:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789762826.637109:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231200/0xdf5a7ac59f1757f8 lrc: 1/0,0 mode: --/PW res: [0x240000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637113:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x10:0x0].0x0 (ffff9c37db743400) 00010000:00010000:3.0:1789762826.637115:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231200/0xdf5a7ac59f1757f8 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637118:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762826.637123:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1757f1 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.637128:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762826.637129:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1757f1 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x10:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.637131:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762826.637134:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762826.637148:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1757e3 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637152:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1757e3 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637156:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1757e3 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637160:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1757e3 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637163:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762826.637171:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1757e3 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637176:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1757e3 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637181:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:1.0:1789762826.637182:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762826.637184:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1757e3 lrc: 3/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637188:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1757e3 lrc: 0/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762826.637194:0:6418:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ed11c27f reply state 0000000032114736 transno 4294967396 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1757ea lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637201:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1757ea lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.637206:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x2b0e463d (ffff9c37e125ad00) 00010000:00010000:1.0:1789762826.637207:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1789762826.637209:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762826.637242:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:13978:x1876702067374592:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1205us (1285us total) trans 4294967396 rc 0/0 00000100:00100000:1.0:1789762826.637248:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 227 00000100:00100000:0.0:1789762826.638103:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762826.638133:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067374848 00000100:00100000:3.0:1789762826.638138:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067374848 00000100:00100000:3.0:1789762826.638160:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 61 00000100:00100000:3.0:1789762826.638166:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:13978:x1876702067374848:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:3.0:1789762826.638254:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:13978:x1876702067374848:12345-192.168.203.37@tcp:35:rm.0 Request processed in 86us (150us total) trans 4294967397 rc 0/0 00000100:00100000:3.0:1789762826.638265:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 61 00000100:00100000:2.0:1789762826.638907:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762826.639002:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067375104 00000100:00100000:1.0:1789762826.639008:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067375104 00000100:00100000:1.0:1789762826.639037:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 228 00000100:00100000:1.0:1789762826.639044:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:13978:x1876702067375104:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:1.0:1789762826.639064:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1789762826.639080:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7e00/0xdf5a7ac59f1757a4 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xf:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c775 expref: 35 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639093:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10af000/0xdf5a7ac59f1756cb lrc: 0/0,0 mode: --/PR res: [0x240000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6e9 expref: 34 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639100:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xf:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:1.0:1789762826.639105:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762826.639113:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230600/0xdf5a7ac59f1757ff lrc: 3/0,1 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639119:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230600/0xdf5a7ac59f1757ff lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639125:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230600/0xdf5a7ac59f1757ff lrc: 3/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639131:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231e00/0xdf5a7ac59f175806 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xe:0x0].0x73550ecb bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639136:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231e00/0xdf5a7ac59f175806 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639140:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231e00/0xdf5a7ac59f175806 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639173:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230000/0xdf5a7ac59f17580d lrc: 3/0,1 mode: --/EX res: [0x240000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639179:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230000/0xdf5a7ac59f17580d lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639191:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230000/0xdf5a7ac59f17580d lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789762826.639447:0:7744:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].9, 64 off8768 00000040:00080000:1.0:1789762826.639458:0:7744:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].9, 64 off8768 00010000:00010000:1.0:1789762826.639486:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175814 lrc: 3/0,1 mode: --/PW res: [0x240000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639493:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175814 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xf:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639498:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175814 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xf:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639503:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175814 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0xf:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639506:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175814 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0xf:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639511:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175814 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0xf:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639526:0:7744:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762826.639534:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230000/0xdf5a7ac59f17580d lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639538:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230000/0xdf5a7ac59f17580d lrc: 2/0,0 mode: EX/EX res: [0x240000402:0xf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639542:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230000/0xdf5a7ac59f17580d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639554:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762826.639556:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230000/0xdf5a7ac59f17580d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639561:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230000/0xdf5a7ac59f17580d lrc: 4/0,0 mode: EX/EX res: [0x240000402:0xf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639567:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xf:0x0].0x0 (ffff9c37db743c00) 00010000:00010000:1.0:1789762826.639569:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762826.639571:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230000/0xdf5a7ac59f17580d lrc: 2/0,0 mode: --/EX res: [0x240000402:0xf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639575:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230000/0xdf5a7ac59f17580d lrc: 0/0,0 mode: --/EX res: [0x240000402:0xf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639586:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230600/0xdf5a7ac59f1757ff lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.639586:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762826.639594:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175814 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0xf:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639595:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230600/0xdf5a7ac59f1757ff lrc: 4/0,1 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.639610:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762826.639611:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230600/0xdf5a7ac59f1757ff lrc: 3/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.639617:0:7217:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175814 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0xf:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639619:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230600/0xdf5a7ac59f1757ff lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639635:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762826.639635:0:7217:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175814 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0xf:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639637:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230600/0xdf5a7ac59f1757ff lrc: 4/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.639665:0:7217:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xf:0x0].0x0 (ffff9c37db743c00) 00010000:00010000:3.0:1789762826.639671:0:7217:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789762826.639676:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230600/0xdf5a7ac59f1757ff lrc: 5/0,0 mode: CW/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639687:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:1.0:1789762826.639689:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762826.639691:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175814 lrc: 1/0,0 mode: --/PW res: [0x240000402:0xf:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639692:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230600/0xdf5a7ac59f1757ff lrc: 3/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.639707:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231000/0xdf5a7ac59f175814 lrc: 0/0,0 mode: --/PW res: [0x240000402:0xf:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.639709:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9230600/0xdf5a7ac59f1757ff lrc: 0/0,0 mode: --/CW res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762826.639728:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e74abed8 reply state 00000000d2da9283 transno 4294967398 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231e00/0xdf5a7ac59f175806 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.639732:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762826.639736:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762826.639737:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231e00/0xdf5a7ac59f175806 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.639738:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762826.639745:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x73550ecb (ffff9c37d3beee00) 00010000:00010000:1.0:1789762826.639747:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1789762826.639749:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762826.639782:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:13978:x1876702067375104:12345-192.168.203.37@tcp:36:rm.0 Request processed in 738us (876us total) trans 4294967398 rc 0/0 00000100:00100000:1.0:1789762826.639787:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 228 00000100:00100000:0.0:1789762826.641712:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762826.641757:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067375232 00000100:00100000:2.0:1789762826.641762:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067375232 00000100:00100000:2.0:1789762826.641795:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 39 00000100:00100000:2.0:1789762826.641801:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067375232:12345-192.168.203.37@tcp:103:rm.0 00010000:00010000:2.0:1789762826.641809:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762826.641822:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f17562a lrc: 0/0,0 mode: --/PR res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6b1 expref: 7 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.641834:0:6404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37e125ab00) 00010000:00010000:2.0:1789762826.641841:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762826.641867:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762826.641873:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067375232:12345-192.168.203.37@tcp:103:rm.0 Request processed in 70us (161us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762826.641879:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 39 00000100:00100000:3.0:1789762826.641905:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067375488 00000100:00100000:3.0:1789762826.641911:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067375488 00000100:00100000:3.0:1789762826.641979:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 62 00000100:00100000:3.0:1789762826.642031:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:13978:x1876702067375488:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:3.0:1789762826.642132:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:13978:x1876702067375488:12345-192.168.203.37@tcp:35:rm.0 Request processed in 101us (264us total) trans 4294967399 rc 0/0 00000100:00100000:3.0:1789762826.642140:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 62 00000100:00100000:1.0:1789762826.642879:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762826.642914:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067375744 00000100:00100000:2.0:1789762826.642941:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067375744 00000100:00100000:2.0:1789762826.642968:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 229 00000100:00100000:2.0:1789762826.642974:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee760380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:13978:x1876702067375744:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:2.0:1789762826.642994:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1789762826.643004:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92c00/0xdf5a7ac59f1757d5 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xe:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c78a expref: 36 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.643027:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94200/0xdf5a7ac59f1756af lrc: 0/0,0 mode: --/PR res: [0x240000402:0xe:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6e2 expref: 35 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.643034:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:2.0:1789762826.643036:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762826.643064:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.643070:0:6419:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.643076:0:6419:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762826.643091:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184d880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074673152:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1789762826.643122:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762826.643146:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074673152 00000100:00100000:3.0:1789762826.643151:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074673152 00000100:00100000:2.0:1789762826.643157:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5fae840 going to sleep for 11 seconds 00000100:00100000:3.0:1789762826.643186:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 271 00000100:00100000:3.0:1789762826.643196:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6419:x1876702074673152:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1789762826.643205:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762826.643218:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93400/0xdf5a7ac59f175822 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f17581b expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.643238:0:6424:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af600/0xdf5a7ac59f17563f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6bf expref: 5 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.643246:0:6424:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93400/0xdf5a7ac59f175822 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f17581b expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.643274:0:6424:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af600/0xdf5a7ac59f17563f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6bf expref: 5 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.643286:0:6424:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af600/0xdf5a7ac59f17563f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6bf expref: 5 pid: 7744 timeout: 399 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762826.643298:0:6424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9f7800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074673280:192.168.203.37@tcp:104: 00000100:00100000:3.0:1789762826.643349:0:6424:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176ff00 going to sleep for 11 seconds 00000100:00100000:3.0:1789762826.644070:0:6424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9f7800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074673280:192.168.203.37@tcp:104: 00010000:00010000:3.0:1789762826.644092:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93400/0xdf5a7ac59f175822 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17581b expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.644102:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762826.644105:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762826.644109:0:6424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93400/0xdf5a7ac59f175822 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17581b expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.644118:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d93400, rc 0) 00000100:00100000:3.0:1789762826.644149:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6419:x1876702074673152:12345-0@lo:101:mdt00_001.0 Request processed in 956us (1026us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762826.644154:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 271 00000100:00100000:2.0:1789762826.644185:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184d880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074673152:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1789762826.644200:0:6419:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.644212:0:6419:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762826.644336:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762826.644387:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067375872 00000100:00100000:0.0:1789762826.644392:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067375872 00010000:00010000:0.0:1789762826.644410:0:7218:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af600/0xdf5a7ac59f17563f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6bf expref: 6 pid: 7744 timeout: 399 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762826.644434:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 73 00000100:00100000:0.0:1789762826.644440:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee760a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067375872:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762826.644447:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762826.644452:0:7218:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af600/0xdf5a7ac59f17563f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6bf expref: 6 pid: 7744 timeout: 399 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.644461:0:7218:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af600/0xdf5a7ac59f17563f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6bf expref: 6 pid: 7744 timeout: 399 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.644471:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d10af600/0xdf5a7ac59f17563f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c6bf expref: 6 pid: 7744 timeout: 399 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.644480:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762826.644484:0:7218:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93400/0xdf5a7ac59f175822 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17581b expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.644490:0:7218:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93400/0xdf5a7ac59f175822 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f17581b expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.644497:0:7218:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93400/0xdf5a7ac59f175822 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f17581b expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.644512:0:7218:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93400/0xdf5a7ac59f175822 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17581b expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762826.644526:0:7218:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:7218:1876702074673408:0@lo:105: 00000100:00100000:0.0:1789762826.644550:0:7218:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762826.644584:0:6449:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074673408 00000100:00100000:0.0:1789762826.644585:0:7218:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c9a9d000 going to sleep for 11 seconds 00000100:00100000:3.0:1789762826.644589:0:6449:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074673408 00000100:00100000:3.0:1789762826.644617:0:6449:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 47 00000100:00100000:3.0:1789762826.644622:0:6449:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:7218:x1876702074673408:12345-0@lo:105: 00010000:00010000:3.0:1789762826.644636:0:6449:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.644657:0:6449:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.644666:0:6449:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.644672:0:6449:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.644677:0:6449:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.644683:0:6449:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.644698:0:6449:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c37c9bc6600) 00010000:00010000:2.0:1789762826.644716:0:6419:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762826.644729:0:6449:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:7218:x1876702074673408:12345-0@lo:105: Request processed in 106us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762826.644735:0:6449:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 47 00010000:00010000:2.0:1789762826.644745:0:6419:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762826.644758:0:7218:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:7218:1876702074673408:0@lo:105: 00000004:00080000:2.0:1789762826.644761:0:6419:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1789762826.644771:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762826.644787:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.644794:0:6419:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.644800:0:6419:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762826.644804:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee760a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067375872:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 362us (467us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762826.644814:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 73 00000100:00100000:2.0:1789762826.644819:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184d880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074673536:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1789762826.644847:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762826.644873:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5fae840 going to sleep for 11 seconds 00000100:00100000:3.0:1789762826.644880:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074673536 00000100:00100000:3.0:1789762826.644885:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074673536 00000100:00100000:3.0:1789762826.644994:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 272 00000100:00100000:3.0:1789762826.645001:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6419:x1876702074673536:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1789762826.645008:0:7227:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762826.645016:0:7227:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175830 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f175829 expref: -99 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.645034:0:7227:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175830 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f175829 expref: 8 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.645042:0:7227:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175830 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f175829 expref: 8 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.645048:0:7227:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba100) 00010000:00010000:3.0:1789762826.645051:0:7227:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762826.645053:0:7227:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d92400, rc 0) 00000100:00100000:3.0:1789762826.645091:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074673536:12345-0@lo:101:mdt00_001.0 Request processed in 90us (242us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762826.645099:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 272 00000100:00100000:2.0:1789762826.645106:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184d880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074673536:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1789762826.645117:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f175830 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.645125:0:6419:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175830 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762826.645134:0:6419:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1789762826.645156:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184d880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074673664:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762826.645176:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762826.645191:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5fae540 going to sleep for 11 seconds 00000100:00100000:0.0:1789762826.645207:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074673664 00000100:00100000:0.0:1789762826.645212:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074673664 00000100:00100000:0.0:1789762826.645241:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 273 00000100:00100000:0.0:1789762826.645248:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074673664:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789762826.645299:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074673664:12345-0@lo:1000:mdt00_001.0 Request processed in 52us (121us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762826.645304:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 273 00000100:00100000:2.0:1789762826.645333:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184d880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074673664:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762826.645381:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074673792:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762826.645400:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762826.645416:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5fae540 going to sleep for 11 seconds 00000100:00100000:3.0:1789762826.645419:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074673792 00000100:00100000:3.0:1789762826.645424:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074673792 00000100:00100000:3.0:1789762826.645447:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 274 00000100:00100000:3.0:1789762826.645452:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074673792:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762826.645607:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074673792:12345-0@lo:1000:mdt00_001.0 Request processed in 153us (204us total) trans 0 rc 1/1 00000100:00100000:3.0:1789762826.645612:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 274 00000100:00100000:2.0:1789762826.645629:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074673792:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1789762826.645673:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 3/0,1 mode: --/EX res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.645679:0:6419:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 3/0,1 mode: --/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.645684:0:6419:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 3/0,1 mode: --/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762826.645692:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074673920:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1789762826.645708:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762826.645722:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5fae840 going to sleep for 11 seconds 00000100:00100000:3.0:1789762826.645729:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074673920 00000100:00100000:3.0:1789762826.645733:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074673920 00000100:00100000:3.0:1789762826.645753:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 275 00000100:00100000:3.0:1789762826.645758:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074673920:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1789762826.645764:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762826.645772:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f17583e lrc: 2/0,0 mode: --/EX res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f175837 expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.645793:0:6424:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f17583e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f175837 expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.645800:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f17583e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f175837 expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.645805:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba600) 00010000:00010000:3.0:1789762826.645807:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762826.645809:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d92800, rc 0) 00000100:00100000:3.0:1789762826.645835:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6419:x1876702074673920:12345-0@lo:101:mdt00_001.0 Request processed in 78us (126us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762826.645841:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 275 00000100:00100000:2.0:1789762826.645854:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074673920:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1789762826.645864:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f17583e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.645870:0:6419:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17583e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.645883:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f175845 lrc: 3/0,1 mode: --/EX res: [0x240000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.645889:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f175845 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.645902:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f175845 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789762826.646379:0:6419:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].8, 424 off38400 00000040:00080000:2.0:1789762826.646418:0:6419:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].8, 504 off39328 00000004:00080000:2.0:1789762826.646423:0:6419:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37f5fae540 with version 17 00000004:00080000:0.0:1789762826.646459:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 17 rpc_version 17 00000020:00100000:0.0:1789762826.646476:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762826.646481:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37d8c8f100 x1876702074674176/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1744/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762826.646492:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8f100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074674176:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762826.646515:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762826.646531:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c9a9d000 going to sleep for 11 seconds 00000100:00100000:2.0:1789762826.646535:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074674176 00000100:00100000:2.0:1789762826.646541:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074674176 00000100:00100000:2.0:1789762826.646566:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:2.0:1789762826.646571:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7224:x1876702074674176:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762826.646951:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7224:x1876702074674176:12345-0@lo:1000:osp_up0-1.0 Request processed in 377us (433us total) trans 4294967386 rc 0/0 00000100:00100000:2.0:1789762826.646959:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00000100:00100000:0.0:1789762826.646988:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8f100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074674176:0@lo:1000:osp_up0-1.0 00000004:00200000:2.0:1789762826.647002:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe:0x0]: returning size 16384 00010000:00010000:2.0:1789762826.647012:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f175845 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647017:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f175845 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647022:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f175845 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647025:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762826.647028:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f175845 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647032:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f175845 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647047:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:2.0:1789762826.647049:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762826.647051:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f175845 lrc: 2/0,0 mode: --/EX res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647056:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f175845 lrc: 0/0,0 mode: --/EX res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647063:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17583e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647068:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f17583e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647073:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f17583e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647086:0:6419:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1789762826.647091:0:6419:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1789762826.647094:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647099:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647104:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647113:0:6419:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1789762826.647117:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000004:00080000:2.0:1789762826.647121:0:6419:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1789762826.647126:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f17583e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647127:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f175830 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.647131:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762826.647145:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.647146:0:6419:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f175830 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.647166:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6600 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:0.0:1789762826.647173:0:8069:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.647181:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7800 already unused, calling callback (ffffffffc1f70ee0) 00000004:00010000:0.0:1789762826.647183:0:8069:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.647192:0:6408:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f17583e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762826.647206:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee760380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:13978:x1876702067375744:12345-192.168.203.37@tcp:36:rm.0 Request processed in 4229us (4325us total) trans 4294967400 rc 0/0 00010000:00010000:0.0:1789762826.647209:0:8069:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762826.647209:0:6408:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f17583e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762826.647217:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 229 00010000:00010000:0.0:1789762826.647228:0:8069:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1789762826.647253:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.647259:0:6408:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 2/0,0 mode: --/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f17583e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.647262:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc6600/0xdf5a7ac59f17581b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f175822 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762826.647262:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074674304:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762826.647282:0:6408:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1789762826.647287:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762826.647290:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762826.647292:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762826.647303:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789762826.647307:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 1/0,0 mode: --/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f17583e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.647318:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7800/0xdf5a7ac59f175837 lrc: 0/0,0 mode: --/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f17583e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762826.647328:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074674304 00000100:00100000:0.0:1789762826.647332:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074674304 00000100:00100000:2.0:1789762826.647334:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:1.0:1789762826.647343:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762826.647346:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee760e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074674432:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762826.647348:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762826.647355:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762826.647371:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 40 00000100:00100000:2.0:1789762826.647380:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762826.647383:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3424:x1876702074674304:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762826.647392:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1789762826.647396:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074674432 00000100:00100000:3.0:1789762826.647402:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074674432 00010000:00010000:0.0:1789762826.647406:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93400/0xdf5a7ac59f175822 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f17581b expref: 10 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.647430:0:8070:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00000100:00100000:3.0:1789762826.647436:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 41 00010000:00010000:0.0:1789762826.647437:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762826.647446:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3424:x1876702074674432:12345-0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762826.647484:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3424:x1876702074674304:12345-0@lo:103:ldlm_bl.0 Request processed in 104us (180us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762826.647490:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074674304:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762826.647497:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 40 00010000:00010000:3.0:1789762826.647506:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762826.647520:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f17583e lrc: 0/0,0 mode: --/EX res: [0x240000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f175837 expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.647531:0:6405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba600) 00010000:00010000:3.0:1789762826.647536:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762826.647566:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074674432:12345-0@lo:103:ldlm_bl.0 Request processed in 123us (190us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762826.647572:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 41 00000100:00100000:2.0:1789762826.647576:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee760e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074674432:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789762826.752082:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074674560:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762826.752160:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762826.752202:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074674560 00000100:00100000:0.0:1789762826.752210:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074674560 00000100:00100000:0.0:1789762826.752262:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 277 00000100:00100000:0.0:1789762826.752276:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:3426:x1876702074674560:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762826.752320:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762826.752371:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:3426:x1876702074674560:12345-0@lo:41:osp-pre-0-1.0 Request processed in 101us (210us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762826.752377:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 277 00000100:00100000:3.0:1789762826.752419:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074674560:0@lo:41:osp-pre-0-1.0 00010000:00010000:3.0:1789762826.881023:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762826.927634:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762826.927684:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067376000 00000100:00100000:3.0:1789762826.927696:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067376000 00000100:00100000:3.0:1789762826.927744:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 230 00000100:00100000:3.0:1789762826.927754:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:13989:x1876702067376000:12345-192.168.203.37@tcp:101:mkdir.0 00010000:00010000:3.0:1789762826.927772:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762826.927790:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f17584c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c791 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.927846:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92200/0xdf5a7ac59f175853 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.927858:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92200/0xdf5a7ac59f175853 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.927867:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92200/0xdf5a7ac59f175853 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762826.927895:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762826.927897:0:6420:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762826.927912:0:6420:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92200/0xdf5a7ac59f175853 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.928079:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f17584c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c791 expref: 7 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.928096:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92200/0xdf5a7ac59f175853 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c791 expref: 6 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.928109:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762826.928111:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762826.928114:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d92200, rc 0) 00000100:00100000:3.0:1789762826.928173:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13989:x1876702067376000:12345-192.168.203.37@tcp:101:mkdir.0 Request processed in 421us (543us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762826.928180:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 230 00000100:00100000:3.0:1789762826.929225:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762826.929291:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067376256 00000100:00100000:1.0:1789762826.929297:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067376256 00000100:00100000:1.0:1789762826.929332:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 231 00000100:00100000:1.0:1789762826.929341:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:13989:x1876702067376256:12345-192.168.203.37@tcp:36:mkdir.0 00010000:00010000:1.0:1789762826.929373:0:11692:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762826.929392:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92200/0xdf5a7ac59f175853 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c791 expref: 6 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.929415:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762826.929420:0:11692:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762826.929474:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17585a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.929482:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17585a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.929488:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17585a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.929497:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.929504:0:11692:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175830 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f175829 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.929510:0:11692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 4/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.929535:0:11692:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175830 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xdf5a7ac59f175829 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762826.929548:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee760000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11692:1876702074674688:0@lo:104: 00000100:00100000:1.0:1789762826.929582:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762826.929628:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7b7ed80 going to sleep for 11 seconds 00000100:00100000:3.0:1789762826.929629:0:6402:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1789762826.929652:0:6402:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074674688 00000100:00100000:3.0:1789762826.929655:0:6402:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074674688 00000100:00100000:3.0:1789762826.929690:0:6402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 48 00000100:00100000:3.0:1789762826.929697:0:6402:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9662450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:11692:x1876702074674688:12345-0@lo:104: 00010000:00010000:3.0:1789762826.929708:0:6402:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f175830 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.929753:0:6402:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1789762826.929760:0:6402:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9662450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:11692:x1876702074674688:12345-0@lo:104: Request processed in 63us (175us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762826.929767:0:6402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 48 00010000:00010000:2.0:1789762826.929779:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1789762826.929781:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee760000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11692:1876702074674688:0@lo:104: 00010000:00010000:2.0:1789762826.929788:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f175830 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.929799:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7200 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:1.0:1789762826.929805:0:11692:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.929805:0:7217:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xdf5a7ac59f175830 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762826.929813:0:7217:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xdf5a7ac59f175830 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.929854:0:7217:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f175830 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.929859:0:7217:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762826.929882:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f175830 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.929888:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7200/0xdf5a7ac59f175829 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f175830 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762826.929899:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074674816:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762826.929905:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762826.929908:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762826.929909:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762826.929955:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762826.929985:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074674816 00000100:00100000:1.0:1789762826.929989:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074674816 00010000:00010000:1.0:1789762826.930007:0:9878:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175830 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xdf5a7ac59f175829 expref: 8 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762826.930030:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 74 00000100:00100000:1.0:1789762826.930038:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3427:x1876702074674816:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762826.930046:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1789762826.939158:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967386 is committed 00010000:00010000:1.0:1789762826.940589:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175830 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xdf5a7ac59f175829 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.940601:0:9878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba100) 00010000:00010000:1.0:1789762826.940605:0:9878:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.940611:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.940616:0:9878:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.940632:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762826.940663:0:11692:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.940672:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762826.940805:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074674816:12345-0@lo:103:ldlm_bl.0 Request processed in 10769us (10849us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762826.940812:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 74 00000100:00100000:0.0:1789762826.940837:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074674816:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789762826.941597:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f175868 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.941610:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f175868 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.941619:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f175868 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762826.941631:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000227ed014 reply state 0000000008102dbc transno 4294967387 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f175868 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.941639:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f175868 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.941656:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:3.0:1789762826.941659:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1789762826.941661:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762826.941664:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 16384 00010000:00010000:3.0:1789762826.941670:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17585a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.941675:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17585a lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.941680:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17585a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.941684:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762826.941687:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17585a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.941701:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17585a lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.941708:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762826.941709:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762826.941711:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17585a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.941715:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f17585a lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762826.941724:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000227ed014 reply state 0000000008102dbc transno 4294967387 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.941729:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.941733:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba100) 00010000:00010000:3.0:1789762826.941735:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1789762826.941737:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:3.0:1789762826.941783:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13989:x1876702067376256:12345-192.168.203.37@tcp:36:mkdir.0 Request processed in 12444us (12561us total) trans 4294967387 rc 0/0 00000100:00100000:3.0:1789762826.941789:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 231 00000100:00100000:1.0:1789762826.948740:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762826.948779:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067376384 00000100:00100000:0.0:1789762826.948786:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067376384 00000100:00100000:0.0:1789762826.948822:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 232 00000100:00100000:0.0:1789762826.948830:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee760000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13990:x1876702067376384:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762826.948846:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762826.948859:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f17586f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c798 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.948900:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f175876 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.948908:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f175876 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.948985:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f175876 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762826.949005:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762826.949008:0:13389:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762826.949021:0:13389:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f175876 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.949038:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f17586f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c798 expref: 9 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.949049:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f175876 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c798 expref: 8 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.949054:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762826.949057:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762826.949059:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1a95c00, rc 0) 00000100:00100000:0.0:1789762826.949103:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee760000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13990:x1876702067376384:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 272us (364us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762826.949111:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 232 00000100:00100000:2.0:1789762826.950259:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762826.950295:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067376640 00000100:00100000:1.0:1789762826.950302:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067376640 00000100:00100000:1.0:1789762826.950353:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 233 00000100:00100000:1.0:1789762826.950364:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13990:x1876702067376640:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:1.0:1789762826.950375:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762826.950377:0:6418:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1789762826.950389:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95c00/0xdf5a7ac59f175876 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c798 expref: 8 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950400:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762826.950403:0:6418:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762826.950411:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f17587d lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c79f expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950487:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f175884 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950494:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f175884 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950500:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f175884 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950508:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f17588b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950513:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f17588b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950518:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f17588b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950561:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f175892 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950573:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f175892 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950579:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f175892 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762826.950586:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 16384 00010000:00010000:1.0:1789762826.950594:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f175892 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950599:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f175892 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950604:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f175892 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950609:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762826.950611:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f175892 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950615:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f175892 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950620:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:1.0:1789762826.950622:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762826.950625:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f175892 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950629:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230c00/0xdf5a7ac59f175892 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950639:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f175884 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950665:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f175884 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950670:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f175884 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950674:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f175884 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950678:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762826.950680:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f175884 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950684:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f175884 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950690:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762826.950692:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762826.950694:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f175884 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950697:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f175884 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950704:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f17588b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950709:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f17588b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950713:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f17588b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950722:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762826.950725:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f17588b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950729:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f17588b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950734:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba100) 00010000:00010000:1.0:1789762826.950736:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762826.950738:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f17588b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950741:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f17588b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950758:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f17587d lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c79f expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950764:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762826.950765:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762826.950768:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f17587d lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c79f expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.950774:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d9230400, rc 0) 00000100:00100000:1.0:1789762826.950804:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13990:x1876702067376640:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 445us (549us total) trans 0 rc 301/301 00000100:00100000:1.0:1789762826.950810:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 233 00000100:00100000:2.0:1789762826.951597:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762826.951632:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067376768 00000100:00100000:1.0:1789762826.951654:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067376768 00000100:00100000:1.0:1789762826.951682:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 234 00000100:00100000:1.0:1789762826.951688:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:13990:x1876702067376768:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:1.0:1789762826.951696:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762826.951703:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f175899 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7a6 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.951727:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1758a0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.951734:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1758a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.951740:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1758a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762826.951752:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762826.951754:0:7744:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:1.0:1789762826.951762:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1758a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.951773:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f175899 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7a6 expref: 9 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.951783:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1758a0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7a6 expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.951788:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762826.951790:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762826.951792:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d9230200, rc 0) 00000100:00100000:1.0:1789762826.951824:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13990:x1876702067376768:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 137us (229us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762826.951830:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 234 00000100:00100000:0.0:1789762826.952719:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762826.952774:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067377024 00000100:00100000:2.0:1789762826.952780:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067377024 00000100:00100000:2.0:1789762826.952818:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 235 00000100:00100000:2.0:1789762826.952830:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13990:x1876702067377024:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762826.952841:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762826.952853:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1758a7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7ad expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.952889:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f1758ae lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.952901:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f1758ae lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.952950:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f1758ae lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.952964:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7a00/0xdf5a7ac59f1758b5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.952973:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7a00/0xdf5a7ac59f1758b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.952981:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7a00/0xdf5a7ac59f1758b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762826.953035:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 16384 00010000:00010000:2.0:1789762826.953102:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f1758ae lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953111:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f1758ae lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953118:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f1758ae lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953125:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f1758ae lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953131:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762826.953135:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f1758ae lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953143:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f1758ae lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953165:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762826.953167:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762826.953171:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f1758ae lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953177:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f1758ae lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953188:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7a00/0xdf5a7ac59f1758b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953194:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7a00/0xdf5a7ac59f1758b5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953200:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7a00/0xdf5a7ac59f1758b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953205:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762826.953208:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7a00/0xdf5a7ac59f1758b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953215:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7a00/0xdf5a7ac59f1758b5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953224:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba100) 00010000:00010000:2.0:1789762826.953227:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762826.953230:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7a00/0xdf5a7ac59f1758b5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953237:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7a00/0xdf5a7ac59f1758b5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953274:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1758a7 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7ad expref: 10 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953283:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762826.953286:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762826.953290:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1758a7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7ad expref: 10 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.953298:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc6400, rc 0) 00000100:00100000:2.0:1789762826.953338:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:13990:x1876702067377024:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 511us (619us total) trans 4294967388 rc 301/301 00000100:00100000:2.0:1789762826.953345:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 235 00000100:00100000:2.0:1789762826.954306:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762826.954357:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067377280 00000100:00100000:0.0:1789762826.954364:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067377280 00000100:00100000:0.0:1789762826.954403:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 236 00000100:00100000:0.0:1789762826.954413:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:13990:x1876702067377280:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:0.0:1789762826.954447:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f1758c3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.954466:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f1758c3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.954473:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f1758c3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762826.954589:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dfb99f88 reply state 00000000293dc948 transno 4294967389 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f1758c3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.954599:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f1758c3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.954605:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:0.0:1789762826.954607:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:0.0:1789762826.954610:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762826.954614:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 16384 00000100:00100000:0.0:1789762826.954675:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:13990:x1876702067377280:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 264us (370us total) trans 4294967389 rc 0/0 00000100:00100000:0.0:1789762826.954681:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 236 00000100:00100000:3.0:1789762826.955334:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762826.955378:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067377536 00000100:00100000:0.0:1789762826.955383:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067377536 00000100:00100000:0.0:1789762826.955419:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 63 00000100:00100000:0.0:1789762826.955428:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:13990:x1876702067377536:12345-192.168.203.37@tcp:35:lfs.0 00000100:00100000:0.0:1789762826.955569:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:13990:x1876702067377536:12345-192.168.203.37@tcp:35:lfs.0 Request processed in 142us (236us total) trans 4294967390 rc 0/0 00000100:00100000:0.0:1789762826.955578:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 63 00000100:00100000:0.0:1789762826.959981:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762826.960027:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067377664 00000100:00100000:1.0:1789762826.960033:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067377664 00000100:00100000:1.0:1789762826.960065:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 237 00000100:00100000:1.0:1789762826.960072:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:13991:x1876702067377664:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:1.0:1789762826.960084:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762826.960094:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1758ca lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7b4 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960157:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758d1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960168:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758d1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960174:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758d1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960182:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758d8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960187:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758d8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960191:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758d8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960231:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1758df lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960244:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1758df lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960249:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1758df lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762826.960265:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 16384 00000004:00200000:1.0:1789762826.960267:0:11692:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:1.0:1789762826.960277:0:11692:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1758df lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960284:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758d1 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960288:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758d1 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960293:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758d1 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960297:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762826.960299:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758d1 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960303:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758d1 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960309:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762826.960312:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762826.960314:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758d1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960327:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758d1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960335:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758d8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960340:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758d8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960344:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758d8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960347:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762826.960349:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758d8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960353:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758d8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960358:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba100) 00010000:00010000:1.0:1789762826.960360:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762826.960363:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758d8 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960369:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758d8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960392:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1758ca lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7b4 expref: 14 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960420:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1758df lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7b4 expref: 13 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.960426:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:1.0:1789762826.960428:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762826.960430:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d9230a00, rc 0) 00000100:00100000:1.0:1789762826.960460:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:13991:x1876702067377664:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 389us (481us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762826.960465:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 237 00000100:00100000:2.0:1789762826.961434:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762826.961474:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067377920 00000100:00100000:0.0:1789762826.961486:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067377920 00000100:00100000:0.0:1789762826.961515:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 238 00000100:00100000:0.0:1789762826.961522:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:13991:x1876702067377920:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:0.0:1789762826.961544:0:13389:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762826.961555:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230a00/0xdf5a7ac59f1758df lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7b4 expref: 13 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.961566:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:0.0:1789762826.961569:0:13389:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762826.961579:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f1758e6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.961586:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f1758e6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.961593:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f1758e6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762826.961716:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005aa611c5 reply state 0000000037617be9 transno 4294967391 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f1758e6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.961725:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f1758e6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762826.961730:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:0.0:1789762826.961732:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:0.0:1789762826.961734:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762826.961738:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 16384 00000100:00100000:0.0:1789762826.961779:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13991:x1876702067377920:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 258us (348us total) trans 4294967391 rc 0/0 00000100:00100000:0.0:1789762826.961793:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 238 00000100:00100000:0.0:1789762826.979978:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762826.980007:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067378048 00000100:00100000:1.0:1789762826.980014:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067378048 00000100:00100000:1.0:1789762826.980050:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 239 00000100:00100000:1.0:1789762826.980057:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:13997:x1876702067378048:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762826.980071:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762826.980082:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1758ed lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7bb expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980120:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758f4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980130:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758f4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980137:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758f4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980144:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758fb lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980156:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758fb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980160:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758fb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980213:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230000/0xdf5a7ac59f175902 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980218:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230000/0xdf5a7ac59f175902 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980222:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230000/0xdf5a7ac59f175902 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762826.980239:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 16384 00000004:00200000:1.0:1789762826.980241:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:1.0:1789762826.980255:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230000/0xdf5a7ac59f175902 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980264:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758f4 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980268:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758f4 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980273:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758f4 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980277:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758f4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980288:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762826.980291:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758f4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980296:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758f4 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980304:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762826.980306:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762826.980309:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758f4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980314:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230800/0xdf5a7ac59f1758f4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980321:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758fb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980325:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758fb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980329:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758fb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980333:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762826.980334:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758fb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980338:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758fb lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980349:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba100) 00010000:00010000:1.0:1789762826.980351:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762826.980353:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758fb lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980356:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230400/0xdf5a7ac59f1758fb lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980376:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9231c00/0xdf5a7ac59f1758ed lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7bb expref: 16 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980389:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230000/0xdf5a7ac59f175902 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7bb expref: 15 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762826.980396:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:1.0:1789762826.980399:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1789762826.980402:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d9230000, rc 0) 00000100:00100000:1.0:1789762826.980442:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:13997:x1876702067378048:12345-192.168.203.37@tcp:101:dd.0 Request processed in 385us (468us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762826.980447:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 239 00000100:00100000:1.0:1789762826.981523:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762826.981576:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067378304 00000100:00100000:2.0:1789762826.981583:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067378304 00000100:00100000:2.0:1789762826.981620:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 240 00000100:00100000:2.0:1789762826.981630:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee760e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:13997:x1876702067378304:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762826.981652:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762826.981665:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7a00/0xdf5a7ac59f175909 lrc: 2/0,0 mode: --/CW res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7c2 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.981740:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f175910 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.981752:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f175910 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.981768:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f175910 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.981783:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175917 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.981791:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175917 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.981798:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175917 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762826.982290:0:7744:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x6:0x0], index 0: 0 00010000:00010000:2.0:1789762826.982429:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6200/0xdf5a7ac59f17591e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982439:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6200/0xdf5a7ac59f17591e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982447:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6200/0xdf5a7ac59f17591e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982474:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f175910 lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982482:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f175910 lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982488:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f175910 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982494:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f175910 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982506:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762826.982509:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f175910 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982516:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f175910 lrc: 5/0,0 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982526:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:2.0:1789762826.982528:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762826.982532:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f175910 lrc: 3/0,0 mode: --/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982538:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f175910 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762826.982549:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ec38b20d reply state 00000000904c9e4d transno 4294967392 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175917 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982559:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175917 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982565:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x4009f790 (ffff9c37d3bee000) 00010000:00010000:2.0:1789762826.982568:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762826.982571:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762826.982593:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7a00/0xdf5a7ac59f175909 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7c2 expref: 18 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982606:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6200/0xdf5a7ac59f17591e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7c2 expref: 17 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762826.982625:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff9c37d3beef00) 00010000:00010000:2.0:1789762826.982628:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762826.982630:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc6200, rc 0) 00000100:00100000:2.0:1789762826.982681:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee760e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13997:x1876702067378304:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1053us (1157us total) trans 4294967392 rc 0/0 00000100:00100000:2.0:1789762826.982688:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 240 00000100:00100000:2.0:1789762826.985995:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762826.986038:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067378432 00000100:00100000:3.0:1789762826.986044:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067378432 00000100:00100000:3.0:1789762826.986077:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299773757951 time 299773836767 state 0x0 ref 1 00000100:00100000:3.0:1789762826.986087:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 299823836767 time 299773836767 state 0x1 ref 1 00000100:00100000:3.0:1789762826.986115:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 2 deadline 299823836767 time 300233193025 state 0x0 ref 1 00000100:00100000:3.0:1789762826.986124:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2397:x1876702067378432:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762826.986151:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762826.986157:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f17592c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.986167:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:3.0:1789762826.986170:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762826.986174:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f17592c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762826.986183:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f17592c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762827.012349:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 0 -> 1048576 00000001:00080000:3.0:1789762827.012349:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967689 is committed 00002000:00010000:2.0:1789762827.012356:0:14170:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb mtime from disk: 0 -> 1789762826 00002000:00010000:2.0:1789762827.012358:0:14170:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb ctime from disk: 0 -> 1789762826 00002000:00010000:2.0:1789762827.012361:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:2.0:1789762827.012374:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f17592c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.012388:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f17592c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.012397:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f17592c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762827.012400:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967691 is committed 00010000:00010000:2.0:1789762827.012406:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762827.012410:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f17592c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.012420:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f17592c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.012455:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f17592c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.012465:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f17592c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762827.012552:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067378432:12345-192.168.203.37@tcp:4:dd.0 Request processed in 26424us (26554us total) trans 4294967691 rc 0/0 00000100:00100000:2.0:1789762827.012563:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 262 00000100:00100000:2.0:1789762827.012573:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 2 deadline 299823836767 time 300233193025 state 0x2 ref 0 00000100:00100000:1.0:1789762827.025795:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762827.025854:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067378560 00000100:00100000:0.0:1789762827.025863:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067378560 00000100:00080000:2.0:1789762827.025935:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762827.025944:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 2 deadline 299823836767 time 300233193025 state 0x0 ref 1 00000100:00100000:0.0:1789762827.025957:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 2 deadline 300283193025 time 300233193025 state 0x1 ref 1 00000100:00100000:0.0:1789762827.025977:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 1 deadline 300283193025 time 300273054876 state 0x0 ref 1 00000100:00100000:0.0:1789762827.025986:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067378560:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762827.026018:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 1048576, mtime: 0x6aad9d0a, atime: 0x0, ctime: 0x6aad9d0a, blocks: 0x803 00010000:00010000:0.0:1789762827.026024:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175933 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.026035:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:0.0:1789762827.026039:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762827.026043:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175933 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.026053:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175933 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762827.038210:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967692 is committed 00002000:00010000:0.0:1789762827.038217:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:0.0:1789762827.038223:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:0.0:1789762827.038238:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175933 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.038248:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175933 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.038255:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175933 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.038260:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762827.038263:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175933 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.038270:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175933 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.038284:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175933 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.038289:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175933 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762827.038342:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067378560:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12355us (12547us total) trans 4294967692 rc 0/0 00000100:00100000:0.0:1789762827.038349:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 263 00000100:00100000:0.0:1789762827.038358:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 1 deadline 300283193025 time 300273054876 state 0x2 ref 0 00000100:00100000:3.0:1789762827.040619:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762827.040679:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067378688 00000100:00100000:0.0:1789762827.040687:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067378688 00000100:00100000:0.0:1789762827.040727:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 1 deadline 300283193025 time 300273054876 state 0x0 ref 1 00000100:00100000:0.0:1789762827.040741:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 1 deadline 300323054876 time 300273054876 state 0x1 ref 1 00000100:00100000:0.0:1789762827.040765:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300323054876 time 300287843653 state 0x0 ref 1 00000100:00100000:0.0:1789762827.040775:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067378688:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762827.040801:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 2097152, mtime: 0x6aad9d0a, atime: 0x0, ctime: 0x6aad9d0a, blocks: 0x1007 00010000:00010000:0.0:1789762827.040806:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17593a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.040817:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:0.0:1789762827.040819:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762827.040824:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17593a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.040844:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17593a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762827.052942:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967693 is committed 00002000:00010000:2.0:1789762827.052952:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:2.0:1789762827.052957:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:2.0:1789762827.052970:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17593a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.052985:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17593a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.052996:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17593a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.053004:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762827.053009:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17593a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.053027:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17593a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.053047:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17593a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.053052:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f17593a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762827.053127:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067378688:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12350us (12506us total) trans 4294967693 rc 0/0 00000100:00100000:2.0:1789762827.053135:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 264 00000100:00100000:2.0:1789762827.053145:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300323054876 time 300287843653 state 0x2 ref 0 00000100:00100000:3.0:1789762827.055536:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762827.055612:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067378816 00000100:00100000:0.0:1789762827.055621:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067378816 00000100:00100000:0.0:1789762827.055677:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300323054876 time 300287843653 state 0x0 ref 1 00000100:00100000:0.0:1789762827.055698:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300337843653 time 300287843653 state 0x1 ref 1 00000100:00100000:2.0:1789762827.090847:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300337843653 time 300337915984 state 0x0 ref 1 00000100:00100000:2.0:1789762827.090865:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067378816:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762827.090907:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 3145728, mtime: 0x6aad9d0a, atime: 0x0, ctime: 0x6aad9d0a, blocks: 0x1807 00010000:00010000:2.0:1789762827.090914:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175941 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.090953:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:2.0:1789762827.090957:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762827.090962:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175941 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.090984:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175941 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762827.103675:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967694 is committed 00002000:00010000:2.0:1789762827.103691:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:2.0:1789762827.103697:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:2.0:1789762827.103709:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175941 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.103723:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175941 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.103734:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175941 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.103744:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762827.103748:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175941 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.103758:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175941 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.103776:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175941 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.103785:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175941 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762827.103861:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067378816:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12998us (48324us total) trans 4294967694 rc 0/0 00000100:00100000:2.0:1789762827.103868:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 265 00000100:00100000:2.0:1789762827.103877:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300337843653 time 300337915984 state 0x2 ref 0 00000100:00100000:1.0:1789762827.106269:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762827.106311:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067378944 00000100:00100000:3.0:1789762827.106319:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067378944 00000100:00100000:3.0:1789762827.106357:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300337843653 time 300337915984 state 0x0 ref 1 00000100:00100000:3.0:1789762827.106376:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300387915984 time 300337915984 state 0x1 ref 1 00000100:00100000:2.0:1789762827.140940:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300387915984 time 300387996911 state 0x0 ref 1 00000100:00100000:2.0:1789762827.140958:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067378944:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762827.140995:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 4194304, mtime: 0x6aad9d0a, atime: 0x0, ctime: 0x6aad9d0a, blocks: 0x2007 00010000:00010000:2.0:1789762827.141001:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175948 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.141014:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:2.0:1789762827.141017:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762827.141022:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175948 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.141032:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175948 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762827.154341:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967695 is committed 00002000:00010000:2.0:1789762827.154350:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:2.0:1789762827.154356:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:2.0:1789762827.154367:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175948 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.154389:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175948 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.154396:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175948 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.154401:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762827.154404:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175948 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.154410:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175948 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.154424:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175948 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.154429:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175948 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762827.154490:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067378944:12345-192.168.203.37@tcp:4:dd.0 Request processed in 13534us (48220us total) trans 4294967695 rc 0/0 00000100:00100000:2.0:1789762827.154499:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 266 00000100:00100000:2.0:1789762827.154509:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300387915984 time 300387996911 state 0x2 ref 0 00000100:00100000:3.0:1789762827.156869:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762827.156904:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067379072 00000100:00100000:2.0:1789762827.156931:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067379072 00000100:00100000:2.0:1789762827.156966:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300387915984 time 300387996911 state 0x0 ref 1 00000100:00100000:2.0:1789762827.156976:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300437996911 time 300387996911 state 0x1 ref 1 00000100:00100000:2.0:1789762827.190990:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300437996911 time 300438055087 state 0x0 ref 1 00000100:00100000:2.0:1789762827.191015:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067379072:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762827.191065:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 5242880, mtime: 0x6aad9d0a, atime: 0x0, ctime: 0x6aad9d0a, blocks: 0x2807 00010000:00010000:2.0:1789762827.191072:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17594f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.191086:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:2.0:1789762827.191090:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762827.191095:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17594f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.191106:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17594f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762827.203395:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967696 is committed 00002000:00010000:2.0:1789762827.203405:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:2.0:1789762827.203410:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:2.0:1789762827.203420:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17594f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.203429:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17594f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.203443:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17594f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.203448:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762827.203452:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17594f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.203458:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17594f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.203473:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17594f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.203478:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17594f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762827.203537:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067379072:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12531us (46668us total) trans 4294967696 rc 0/0 00000100:00100000:2.0:1789762827.203546:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 267 00000100:00100000:2.0:1789762827.203554:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300437996911 time 300438055087 state 0x2 ref 0 00000100:00100000:3.0:1789762827.205782:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762827.205824:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067379200 00000100:00100000:2.0:1789762827.205833:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067379200 00000100:00100000:2.0:1789762827.205868:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300437996911 time 300438055087 state 0x0 ref 1 00000100:00100000:2.0:1789762827.205889:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300488055087 time 300438055087 state 0x1 ref 1 00000100:00100000:0.0:1789762827.241093:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300488055087 time 300488162649 state 0x0 ref 1 00000100:00100000:0.0:1789762827.241113:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067379200:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762827.241152:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 6291456, mtime: 0x6aad9d0a, atime: 0x0, ctime: 0x6aad9d0a, blocks: 0x3007 00010000:00010000:0.0:1789762827.241159:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175956 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.241172:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:0.0:1789762827.241175:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762827.241180:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.241189:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762827.252224:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967697 is committed 00002000:00010000:2.0:1789762827.252240:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:2.0:1789762827.252245:0:14171:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb mtime from disk: 1789762826 -> 1789762827 00002000:00010000:2.0:1789762827.252248:0:14171:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb ctime from disk: 1789762826 -> 1789762827 00002000:00010000:2.0:1789762827.252250:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 12295 -> 14343 00010000:00010000:2.0:1789762827.252261:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.252270:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175956 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.252277:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175956 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.252282:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762827.252286:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175956 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.252292:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175956 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.252306:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175956 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.252312:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175956 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762827.252382:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067379200:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11273us (46599us total) trans 4294967697 rc 0/0 00000100:00100000:2.0:1789762827.252391:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 268 00000100:00100000:2.0:1789762827.252400:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300488055087 time 300488162649 state 0x2 ref 0 00000100:00100000:3.0:1789762827.254624:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762827.254673:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067379328 00000100:00100000:2.0:1789762827.254681:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067379328 00000100:00100000:2.0:1789762827.254720:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300488055087 time 300488162649 state 0x0 ref 1 00000100:00100000:2.0:1789762827.254731:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300538162649 time 300488162649 state 0x1 ref 1 00000001:00080000:3.0:1789762827.271798:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967389 is committed 00000001:00080000:3.0:1789762827.271822:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967398 is committed 00010000:00010000:1.0:1789762827.271829:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92e00/0xdf5a7ac59f175600 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762827.271829:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 12 committed 00010000:00010000:1.0:1789762827.271841:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92e00/0xdf5a7ac59f175600 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762827.271843:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 12 committed 00010000:00010000:0.0:1789762827.271848:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1757ea lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.271859:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1757ea lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762827.271863:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967400 is committed 00010000:00010000:1.0:1789762827.271865:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92e00/0xdf5a7ac59f175600 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.271871:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1757ea lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.271874:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d92e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762827.271881:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92e00/0xdf5a7ac59f175600 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.271897:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92e00/0xdf5a7ac59f175600 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.271898:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762827.271905:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1757ea lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1789762827.271909:0:7221:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9c37d176fa80 refcount 2 committed 0 result 0 batchid 7 00010000:00010000:0.0:1789762827.271913:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1757ea lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1789762827.271915:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c36c2126e00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9c37d176fc00 00000020:00080000:3.0:1789762827.271934:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x2:0x0].7 00000020:00080000:3.0:1789762827.271937:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c36c2126900 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9c37d2033000 00010000:00010000:1.0:1789762827.271938:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00000020:00080000:3.0:1789762827.271940:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x2:0x0].7 00010000:00010000:1.0:1789762827.271943:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762827.271949:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92e00/0xdf5a7ac59f175600 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.271956:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92e00/0xdf5a7ac59f175600 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.271957:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x2b0e463d (ffff9c37e125ad00) 00010000:00010000:0.0:1789762827.271964:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762827.271969:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1757ea lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.271974:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231400/0xdf5a7ac59f1757ea lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.271977:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aee00/0xdf5a7ac59f175670 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.271984:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aee00/0xdf5a7ac59f175670 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.271990:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231e00/0xdf5a7ac59f175806 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.271990:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aee00/0xdf5a7ac59f175670 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762827.271995:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074674944:0@lo:6:osp-syn-0-1.0 00010000:00010000:1.0:1789762827.272012:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d10aee00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762827.272013:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231e00/0xdf5a7ac59f175806 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272015:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aee00/0xdf5a7ac59f175670 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.272026:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231e00/0xdf5a7ac59f175806 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.272033:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762827.272038:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231e00/0xdf5a7ac59f175806 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272042:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aee00/0xdf5a7ac59f175670 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.272045:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231e00/0xdf5a7ac59f175806 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272054:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:1.0:1789762827.272061:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:3.0:1789762827.272063:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789762827.272066:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aee00/0xdf5a7ac59f175670 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272075:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10aee00/0xdf5a7ac59f175670 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.272078:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x73550ecb (ffff9c37d3beee00) 00010000:00010000:0.0:1789762827.272082:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762827.272086:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231e00/0xdf5a7ac59f175806 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272091:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175685 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.272093:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d9231e00/0xdf5a7ac59f175806 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762827.272101:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074674944 00010000:00010000:1.0:1789762827.272109:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175685 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762827.272111:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074674944 00010000:00010000:1.0:1789762827.272118:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175685 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272123:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d92a00 already unused, calling callback (ffffffffc1f644f0) 00000020:00080000:0.0:1789762827.272126:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 7 committed batchid 6 00010000:00010000:1.0:1789762827.272126:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175685 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272135:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175685 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789762827.272145:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762827.272146:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:1.0:1789762827.272151:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789762827.272156:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175685 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272175:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37f7d92a00/0xdf5a7ac59f175685 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762827.272183:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:2.0:1789762827.272198:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ffccac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074674944:12345-0@lo:6:osp-syn-0-1.0 00010000:00010000:1.0:1789762827.272205:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10ae400/0xdf5a7ac59f1756c4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272212:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10ae400/0xdf5a7ac59f1756c4 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272217:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10ae400/0xdf5a7ac59f1756c4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272221:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d10ae400 already unused, calling callback (ffffffffc1f644f0) 00002000:00080000:2.0:1789762827.272224:0:8054:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:9 count 1 00010000:00010000:1.0:1789762827.272225:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10ae400/0xdf5a7ac59f1756c4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272234:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10ae400/0xdf5a7ac59f1756c4 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272243:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x73550ecb (ffff9c37d3beee00) 00010000:00010000:1.0:1789762827.272246:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762827.272248:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10ae400/0xdf5a7ac59f1756c4 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1789762827.272253:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 6 00010000:00010000:1.0:1789762827.272253:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d10ae400/0xdf5a7ac59f1756c4 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762827.272259:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x9:0x0] initial LVB size: 8388608, mtime: 0x6aad9d09, atime: 0x0, ctime: 0x6aad9d09, blocks: 0x4007 00010000:00010000:1.0:1789762827.272271:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f1756fc lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.272271:0:8054:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17595d lrc: 3/0,1 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272287:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f1756fc lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.272296:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=300s v=(0 0) 00010000:00010000:1.0:1789762827.272299:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f1756fc lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.272302:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762827.272304:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762827.272309:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f1756fc lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.272314:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17595d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272315:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f1756fc lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272331:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe:0x0].0x2b0e463d (ffff9c37e125ad00) 00010000:00010000:1.0:1789762827.272333:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762827.272335:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f1756fc lrc: 2/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272346:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1a94e00/0xdf5a7ac59f1756fc lrc: 0/0,0 mode: --/TXN res: [0x240000402:0xe:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.272366:0:8054:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17595d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.272376:0:8054:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17595d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.272384:0:8054:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17595d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.272406:0:8054:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762827.272424:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762827.272430:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17595d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.272438:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762827.272442:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17595d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762827.272443:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:0.0:1789762827.272454:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17595d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762827.272457:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff70df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074675072:0@lo:6:osp-syn-1-1.0 00010000:00010000:0.0:1789762827.272473:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17595d lrc: 1/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.272482:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f17595d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762827.272500:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1789762827.272522:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762827.272525:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762827.272531:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762827.272534:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074675072 00000100:00100000:3.0:1789762827.272540:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074675072 00000100:00080000:0.0:1789762827.272567:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762827.272579:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:3.0:1789762827.272587:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074675072:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:3.0:1789762827.272603:0:8053:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:9 count 1 00002000:00010000:3.0:1789762827.272637:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x9:0x0] initial LVB size: 8388608, mtime: 0x6aad9d09, atime: 0x0, ctime: 0x6aad9d09, blocks: 0x1 00010000:00010000:3.0:1789762827.272659:0:8053:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175964 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.272669:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=300s v=(0 0) 00010000:00010000:3.0:1789762827.272672:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762827.272676:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175964 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.272687:0:8053:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175964 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.272693:0:8053:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175964 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.272699:0:8053:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175964 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.272713:0:8053:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762827.272731:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762827.272737:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175964 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272745:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762827.272748:0:6408:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175964 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272764:0:6408:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175964 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272776:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175964 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272783:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175964 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.272800:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762827.272802:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762827.272805:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762827.273108:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ffccac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074674944:12345-0@lo:6:osp-syn-0-1.0 Request processed in 914us (1046us total) trans 4294967698 rc 0/0 00000100:00100000:2.0:1789762827.273119:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000004:00080000:1.0:1789762827.273138:0:3424:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37c96a2a00/2, rc 0, transno 4294967698 00000100:00100000:1.0:1789762827.273160:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074674944:0@lo:6:osp-syn-0-1.0 00000100:00100000:3.0:1789762827.273376:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074675072:12345-0@lo:6:osp-syn-1-1.0 Request processed in 790us (879us total) trans 4294967380 rc 0/0 00000100:00100000:3.0:1789762827.273382:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000004:00080000:1.0:1789762827.273399:0:3424:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37ff70df80/2, rc 0, transno 4294967380 00000100:00100000:1.0:1789762827.273411:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff70df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074675072:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1789762827.291174:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300538162649 time 300538243299 state 0x0 ref 1 00000100:00100000:0.0:1789762827.291190:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067379328:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762827.291222:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 7340032, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x3807 00010000:00010000:0.0:1789762827.291228:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f17596b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.291238:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:0.0:1789762827.291249:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762827.291256:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f17596b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.291266:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f17596b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762827.306890:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967699 is committed 00002000:00010000:2.0:1789762827.306904:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:2.0:1789762827.306909:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 14343 -> 16391 00000001:00080000:3.0:1789762827.306911:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967698 is committed 00010000:00010000:2.0:1789762827.306940:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f17596b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.306952:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f17596b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.306960:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f17596b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.306969:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762827.306981:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f17596b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.306992:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f17596b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.307013:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f17596b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.307019:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f17596b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762827.307078:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067379328:12345-192.168.203.37@tcp:4:dd.0 Request processed in 15890us (52455us total) trans 4294967699 rc 0/0 00000100:00100000:2.0:1789762827.307086:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 269 00000100:00100000:2.0:1789762827.307095:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300538162649 time 300538243299 state 0x2 ref 0 00000100:00100000:3.0:1789762827.309332:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762827.309367:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067379456 00000100:00100000:2.0:1789762827.309375:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067379456 00000100:00100000:2.0:1789762827.309413:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300538162649 time 300538243299 state 0x0 ref 1 00000100:00100000:2.0:1789762827.309424:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300588243299 time 300538243299 state 0x1 ref 1 00000100:00100000:3.0:1789762827.341253:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300588243299 time 300588322706 state 0x0 ref 1 00000100:00100000:3.0:1789762827.341286:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067379456:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762827.341327:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 8388608, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x4007 00010000:00010000:3.0:1789762827.341333:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175972 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.341345:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:3.0:1789762827.341349:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762827.341353:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175972 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.341364:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175972 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762827.351466:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967700 is committed 00002000:00010000:2.0:1789762827.351472:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 8388608 -> 9437184 00002000:00010000:2.0:1789762827.351477:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 16391 -> 18439 00010000:00010000:2.0:1789762827.351489:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175972 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.351508:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175972 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.351515:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175972 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.351520:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762827.351523:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175972 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.351529:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175972 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.351543:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175972 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.351548:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175972 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762827.351602:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067379456:12345-192.168.203.37@tcp:4:dd.0 Request processed in 10319us (42270us total) trans 4294967700 rc 0/0 00000100:00100000:2.0:1789762827.351609:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 270 00000100:00100000:2.0:1789762827.351618:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300588243299 time 300588322706 state 0x2 ref 0 00000100:00100000:3.0:1789762827.353749:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762827.353792:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067379584 00000100:00100000:2.0:1789762827.353805:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067379584 00000100:00100000:2.0:1789762827.353840:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300588243299 time 300588322706 state 0x0 ref 1 00000100:00100000:2.0:1789762827.353851:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300638322706 time 300588322706 state 0x1 ref 1 00000100:00100000:0.0:1789762827.391318:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300638322706 time 300638387905 state 0x0 ref 1 00000100:00100000:0.0:1789762827.391340:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067379584:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762827.391388:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 9437184, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x4807 00010000:00010000:0.0:1789762827.391396:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175979 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.391418:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:0.0:1789762827.391421:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762827.391426:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175979 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.391437:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175979 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762827.402403:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967701 is committed 00002000:00010000:3.0:1789762827.402422:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 9437184 -> 10485760 00002000:00010000:3.0:1789762827.402428:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 18439 -> 20487 00010000:00010000:3.0:1789762827.402438:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175979 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.402448:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175979 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.402455:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175979 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.402471:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762827.402475:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175979 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.402481:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175979 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.402495:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175979 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.402500:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175979 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762827.402557:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067379584:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11223us (48808us total) trans 4294967701 rc 0/0 00000100:00100000:3.0:1789762827.402565:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 271 00000100:00100000:3.0:1789762827.402573:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300638322706 time 300638387905 state 0x2 ref 0 00000100:00100000:3.0:1789762827.404777:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762827.404826:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067379712 00000100:00100000:2.0:1789762827.404835:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067379712 00000100:00100000:2.0:1789762827.404880:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300638322706 time 300638387905 state 0x0 ref 1 00000100:00100000:2.0:1789762827.404896:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300688387905 time 300638387905 state 0x1 ref 1 00000100:00100000:3.0:1789762827.441390:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300688387905 time 300688459128 state 0x0 ref 1 00000100:00100000:3.0:1789762827.441409:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067379712:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762827.441452:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 10485760, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x5007 00010000:00010000:3.0:1789762827.441458:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175980 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.441481:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:3.0:1789762827.441484:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762827.441489:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175980 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.441499:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175980 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762827.452848:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967702 is committed 00002000:00010000:0.0:1789762827.452932:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:0.0:1789762827.452937:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:0.0:1789762827.452948:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175980 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.452958:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175980 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.452965:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175980 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.452970:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d92000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762827.452973:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175980 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.452980:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175980 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.453005:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175980 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.453011:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175980 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762827.453085:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067379712:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11672us (48299us total) trans 4294967702 rc 0/0 00000100:00100000:0.0:1789762827.453093:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 272 00000100:00100000:0.0:1789762827.453104:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300688387905 time 300688459128 state 0x2 ref 0 00000100:00100000:3.0:1789762827.455337:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762827.455410:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067379840 00000100:00100000:2.0:1789762827.455419:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067379840 00000100:00100000:2.0:1789762827.455459:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300688387905 time 300688459128 state 0x0 ref 1 00000100:00100000:2.0:1789762827.455469:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300738459128 time 300688459128 state 0x1 ref 1 00000100:00100000:0.0:1789762827.491472:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300738459128 time 300738537882 state 0x0 ref 1 00000100:00100000:0.0:1789762827.491496:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067379840:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762827.491546:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 11534336, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x5807 00010000:00010000:0.0:1789762827.491553:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175987 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.491566:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:0.0:1789762827.491571:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762827.491577:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175987 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.491593:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175987 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762827.502481:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 11534336 -> 12582912 00002000:00010000:2.0:1789762827.502487:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 22535 -> 24583 00000001:00080000:1.0:1789762827.502488:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967703 is committed 00010000:00010000:2.0:1789762827.502497:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175987 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.502507:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175987 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.502514:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175987 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.502530:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762827.502534:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175987 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.502540:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175987 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.502554:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175987 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.502560:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175987 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762827.502618:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067379840:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11130us (47279us total) trans 4294967703 rc 0/0 00000100:00100000:2.0:1789762827.502625:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 273 00000100:00100000:2.0:1789762827.502635:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300738459128 time 300738537882 state 0x2 ref 0 00000100:00100000:3.0:1789762827.504912:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762827.504967:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067379968 00000100:00100000:0.0:1789762827.504975:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067379968 00000100:00100000:0.0:1789762827.505015:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300738459128 time 300738537882 state 0x0 ref 1 00000100:00100000:0.0:1789762827.505026:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300788537882 time 300738537882 state 0x1 ref 1 00000100:00100000:3.0:1789762827.541557:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300788537882 time 300788618648 state 0x0 ref 1 00000100:00100000:3.0:1789762827.541596:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067379968:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762827.541667:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 12582912, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x6007 00010000:00010000:3.0:1789762827.541676:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f17598e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.541691:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:3.0:1789762827.541696:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762827.541703:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f17598e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.541717:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f17598e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762827.552793:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967704 is committed 00002000:00010000:1.0:1789762827.552833:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 12582912 -> 13631488 00002000:00010000:1.0:1789762827.552836:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 24583 -> 26631 00010000:00010000:1.0:1789762827.552846:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f17598e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.552866:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f17598e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.552873:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f17598e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.552878:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762827.552881:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f17598e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.552888:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f17598e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.552902:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f17598e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.552907:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f17598e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762827.552995:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067379968:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11416us (48079us total) trans 4294967704 rc 0/0 00000100:00100000:1.0:1789762827.553006:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 274 00000100:00100000:1.0:1789762827.553015:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300788537882 time 300788618648 state 0x2 ref 0 00000100:00100000:1.0:1789762827.555213:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762827.555263:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067380096 00000100:00100000:0.0:1789762827.555271:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067380096 00000100:00100000:0.0:1789762827.555308:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300788537882 time 300788618648 state 0x0 ref 1 00000100:00100000:0.0:1789762827.555319:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300838618648 time 300788618648 state 0x1 ref 1 00000100:00100000:0.0:1789762827.591603:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300838618648 time 300838671737 state 0x0 ref 1 00000100:00100000:0.0:1789762827.591624:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067380096:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762827.591682:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 13631488, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x6807 00010000:00010000:0.0:1789762827.591689:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175995 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.591702:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:0.0:1789762827.591706:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762827.591710:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175995 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.591736:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175995 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762827.604944:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967705 is committed 00002000:00010000:1.0:1789762827.605004:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 13631488 -> 14680064 00002000:00010000:1.0:1789762827.605009:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:1.0:1789762827.605019:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175995 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.605028:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175995 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.605034:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175995 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.605039:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762827.605043:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175995 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.605049:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175995 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.605064:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175995 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.605069:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175995 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762827.605140:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067380096:12345-192.168.203.37@tcp:4:dd.0 Request processed in 13519us (49923us total) trans 4294967705 rc 0/0 00000100:00100000:1.0:1789762827.605148:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 275 00000100:00100000:1.0:1789762827.605164:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300838618648 time 300838671737 state 0x2 ref 0 00000100:00100000:3.0:1789762827.607428:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762827.607504:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067380224 00000100:00100000:2.0:1789762827.607512:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067380224 00000100:00100000:2.0:1789762827.607551:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300838618648 time 300838671737 state 0x0 ref 1 00000100:00100000:2.0:1789762827.607562:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300888671737 time 300838671737 state 0x1 ref 1 00000100:00100000:2.0:1789762827.641678:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300888671737 time 300888747160 state 0x0 ref 1 00000100:00100000:2.0:1789762827.641707:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067380224:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762827.641748:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 14680064, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x7007 00010000:00010000:2.0:1789762827.641755:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17599c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.641768:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:2.0:1789762827.641772:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762827.641776:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17599c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.641787:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17599c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762827.654949:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967706 is committed 00002000:00010000:1.0:1789762827.654965:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 14680064 -> 15728640 00002000:00010000:1.0:1789762827.654970:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:1.0:1789762827.654982:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17599c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.654992:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17599c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.654998:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17599c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.655003:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762827.655008:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17599c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.655014:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17599c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.655029:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17599c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.655034:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17599c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762827.655101:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067380224:12345-192.168.203.37@tcp:4:dd.0 Request processed in 13395us (47671us total) trans 4294967706 rc 0/0 00000100:00100000:1.0:1789762827.655116:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 276 00000100:00100000:1.0:1789762827.655125:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300888671737 time 300888747160 state 0x2 ref 0 00000100:00100000:3.0:1789762827.657459:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762827.657498:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067380352 00000100:00100000:1.0:1789762827.657507:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067380352 00000100:00100000:1.0:1789762827.657546:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300888671737 time 300888747160 state 0x0 ref 1 00000100:00100000:1.0:1789762827.657557:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300938747160 time 300888747160 state 0x1 ref 1 00000100:00100000:2.0:1789762827.691755:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300938747160 time 300938822840 state 0x0 ref 1 00000100:00100000:2.0:1789762827.691775:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067380352:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762827.691818:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 15728640, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x7807 00010000:00010000:2.0:1789762827.691825:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.691839:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:2.0:1789762827.691842:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762827.691847:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.691869:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762827.703284:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967707 is committed 00002000:00010000:1.0:1789762827.703288:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 15728640 -> 16777216 00002000:00010000:1.0:1789762827.703294:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 30727 -> 32775 00010000:00010000:1.0:1789762827.703304:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.703314:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.703323:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.703332:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762827.703337:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.703347:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.703380:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.703386:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762827.703443:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067380352:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11673us (45983us total) trans 4294967707 rc 0/0 00000100:00100000:1.0:1789762827.703451:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 277 00000100:00100000:1.0:1789762827.703460:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300938747160 time 300938822840 state 0x2 ref 0 00000100:00100000:3.0:1789762827.705820:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762827.705859:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067380480 00000100:00100000:1.0:1789762827.705867:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067380480 00000100:00100000:1.0:1789762827.705902:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300938747160 time 300938822840 state 0x0 ref 1 00000100:00100000:1.0:1789762827.705913:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300988822840 time 300938822840 state 0x1 ref 1 00000100:00100000:0.0:1789762827.719742:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762827.719773:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762827.719790:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067380608 00000100:00100000:0.0:1789762827.719794:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762827.719798:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067380608 00000100:00100000:2.0:1789762827.719821:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067380736 00000100:00100000:0.0:1789762827.719822:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762827.719829:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067380736 00000100:00100000:3.0:1789762827.719845:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067380864 00000100:00100000:3.0:1789762827.719853:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067380864 00000100:00100000:1.0:1789762827.719856:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 343 00000100:00100000:0.0:1789762827.719858:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762827.719871:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1579050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067380608:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762827.719886:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067381120 00000100:00100000:0.0:1789762827.719890:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762827.719892:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067381120 00000100:00100000:3.0:1789762827.719903:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067381248 00000100:00100000:3.0:1789762827.719909:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067381248 00000100:00100000:0.0:1789762827.719950:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 408 00000100:00100000:1.0:1789762827.719953:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1579050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067380608:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 86us (212us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762827.719963:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 343 00000100:00100000:0.0:1789762827.719966:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:2398:x1876702067380736:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762827.719973:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 409 00000100:00100000:2.0:1789762827.719986:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 410 00000100:00100000:3.0:1789762827.719989:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067380864:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762827.720000:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067381120:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762827.720010:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:2398:x1876702067380736:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 49us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762827.720011:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067380992 00000100:00100000:1.0:1789762827.720017:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067380992 00000100:00100000:0.0:1789762827.720021:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 408 00000100:00100000:3.0:1789762827.720043:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067380864:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 56us (245us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762827.720049:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 411 00000100:00100000:2.0:1789762827.720051:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067381120:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 53us (194us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762827.720061:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 158 00000100:00100000:0.0:1789762827.720065:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067381248:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762827.720067:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 409 00000100:00100000:1.0:1789762827.720075:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2398:x1876702067380992:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762827.720080:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 410 00000100:00100000:0.0:1789762827.720103:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067381248:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 42us (213us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762827.720108:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 411 00000100:00080000:3.0:1789762827.720123:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762827.720125:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2398:x1876702067380992:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 52us (302us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762827.720131:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762827.720135:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 158 00000100:00080000:3.0:1789762827.720137:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762827.741862:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300988822840 time 300988933648 state 0x0 ref 1 00000100:00100000:0.0:1789762827.741879:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067380480:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762827.741951:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 16777216, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x8007 00010000:00010000:0.0:1789762827.741958:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f1759aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.741971:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=300s v=(0 0) 00010000:00010000:0.0:1789762827.741975:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762827.741979:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f1759aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.741990:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f1759aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762827.753990:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 16777216 -> 17825792 00000001:00080000:2.0:1789762827.753992:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967708 is committed 00002000:00010000:1.0:1789762827.753997:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:1.0:1789762827.754009:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f1759aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.754019:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f1759aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.754026:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f1759aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.754031:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762827.754035:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f1759aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.754041:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f1759aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.754058:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f1759aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.754074:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f1759aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762827.754148:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067380480:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12271us (48327us total) trans 4294967708 rc 0/0 00000100:00100000:1.0:1789762827.754156:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 278 00000100:00100000:1.0:1789762827.754165:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300988822840 time 300988933648 state 0x2 ref 0 00000100:00100000:3.0:1789762827.756580:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762827.756614:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067381376 00000100:00100000:1.0:1789762827.756622:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067381376 00000100:00100000:1.0:1789762827.756673:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 300988822840 time 300988933648 state 0x0 ref 1 00000100:00100000:1.0:1789762827.756683:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301038933648 time 300988933648 state 0x1 ref 1 00000100:00100000:0.0:1789762827.791967:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301038933648 time 301039036256 state 0x0 ref 1 00000100:00100000:0.0:1789762827.791988:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067381376:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762827.792031:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 17825792, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x8807 00010000:00010000:0.0:1789762827.792038:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f1759b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.792051:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:0.0:1789762827.792055:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762827.792059:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f1759b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762827.792070:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f1759b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762827.803121:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967709 is committed 00002000:00010000:1.0:1789762827.803133:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 17825792 -> 18874368 00002000:00010000:1.0:1789762827.803139:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 34823 -> 36871 00010000:00010000:1.0:1789762827.803152:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f1759b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.803166:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f1759b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.803175:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f1759b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.803191:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762827.803194:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f1759b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.803201:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f1759b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.803216:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f1759b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.803221:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f1759b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762827.803278:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067381376:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11295us (46698us total) trans 4294967709 rc 0/0 00000100:00100000:1.0:1789762827.803285:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 279 00000100:00100000:1.0:1789762827.803294:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301038933648 time 301039036256 state 0x2 ref 0 00000100:00100000:3.0:1789762827.805574:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762827.805614:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067381504 00000100:00100000:1.0:1789762827.805623:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067381504 00000100:00100000:1.0:1789762827.805671:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301038933648 time 301039036256 state 0x0 ref 1 00000100:00100000:1.0:1789762827.805683:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301089036256 time 301039036256 state 0x1 ref 1 00000100:00100000:2.0:1789762827.842089:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301089036256 time 301089157603 state 0x0 ref 1 00000100:00100000:2.0:1789762827.842108:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067381504:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762827.842149:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 18874368, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x9007 00010000:00010000:2.0:1789762827.842155:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.842168:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:2.0:1789762827.842171:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762827.842187:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.842198:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762827.853627:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967710 is committed 00002000:00010000:1.0:1789762827.853695:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:1.0:1789762827.853706:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:1.0:1789762827.853716:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.853726:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.853733:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.853738:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762827.853741:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.853747:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.853761:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.853766:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1759b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762827.853830:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067381504:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11725us (48255us total) trans 4294967710 rc 0/0 00000100:00100000:1.0:1789762827.853838:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 280 00000100:00100000:1.0:1789762827.853847:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301089036256 time 301089157603 state 0x2 ref 0 00000100:00100000:3.0:1789762827.856056:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762827.856097:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067381632 00000100:00100000:1.0:1789762827.856105:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067381632 00000100:00100000:1.0:1789762827.856150:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301089036256 time 301089157603 state 0x0 ref 1 00000100:00100000:1.0:1789762827.856172:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301139157603 time 301089157603 state 0x1 ref 1 00000100:00100000:3.0:1789762827.892184:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301139157603 time 301139245252 state 0x0 ref 1 00000100:00100000:3.0:1789762827.892209:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067381632:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762827.892256:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 19922944, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0x9807 00010000:00010000:3.0:1789762827.892263:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f1759bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.892290:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:3.0:1789762827.892294:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762827.892298:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f1759bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762827.892310:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f1759bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762827.903540:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967711 is committed 00002000:00010000:1.0:1789762827.903552:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 19922944 -> 20971520 00002000:00010000:1.0:1789762827.903557:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:1.0:1789762827.903566:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f1759bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.903576:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f1759bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.903582:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f1759bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.903587:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762827.903591:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f1759bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.903598:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f1759bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.903613:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f1759bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.903628:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089800f11/0xdf5a7ac59f1759bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762827.903700:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067381632:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11498us (47642us total) trans 4294967711 rc 0/0 00000100:00100000:1.0:1789762827.903708:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 281 00000100:00100000:1.0:1789762827.903717:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301139157603 time 301139245252 state 0x2 ref 0 00010000:00010000:3.0:1789762827.904012:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762827.906032:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762827.906109:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067381760 00000100:00100000:0.0:1789762827.906118:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067381760 00000100:00100000:0.0:1789762827.906154:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301139157603 time 301139245252 state 0x0 ref 1 00000100:00100000:0.0:1789762827.906165:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301189245252 time 301139245252 state 0x1 ref 1 00000100:00100000:2.0:1789762827.942264:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301189245252 time 301189335030 state 0x0 ref 1 00000100:00100000:2.0:1789762827.942280:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067381760:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762827.942321:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 20971520, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0xa007 00010000:00010000:2.0:1789762827.942331:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f1759c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.942347:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:2.0:1789762827.942350:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762827.942355:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f1759c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.942365:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f1759c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762827.953275:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:1.0:1789762827.953282:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 40967 -> 43015 00000001:00080000:2.0:1789762827.953282:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967712 is committed 00010000:00010000:1.0:1789762827.953297:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f1759c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.953307:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f1759c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.953314:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f1759c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.953319:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762827.953323:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f1759c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.953328:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f1759c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.953348:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f1759c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762827.953354:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f1759c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762827.953405:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067381760:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11128us (47373us total) trans 4294967712 rc 0/0 00000100:00100000:1.0:1789762827.953412:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 282 00000100:00100000:1.0:1789762827.953420:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301189245252 time 301189335030 state 0x2 ref 0 00000100:00100000:3.0:1789762827.955599:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762827.955672:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067381888 00000100:00100000:1.0:1789762827.955679:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067381888 00000100:00100000:1.0:1789762827.955714:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301189245252 time 301189335030 state 0x0 ref 1 00000100:00100000:1.0:1789762827.955724:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301239335030 time 301189335030 state 0x1 ref 1 00000100:00100000:2.0:1789762827.992395:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301239335030 time 301239461426 state 0x0 ref 1 00000100:00100000:2.0:1789762827.992420:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067381888:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762827.992482:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 22020096, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0xa807 00010000:00010000:2.0:1789762827.992489:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f1759cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.992502:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:2.0:1789762827.992506:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762827.992510:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f1759cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762827.992521:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f1759cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762828.003547:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967713 is committed 00002000:00010000:1.0:1789762828.003559:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 22020096 -> 23068672 00002000:00010000:1.0:1789762828.003565:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:1.0:1789762828.003575:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f1759cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.003584:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f1759cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.003601:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f1759cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.003606:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.003609:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f1759cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.003615:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f1759cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.003629:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f1759cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.003649:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f1759cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.003705:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067381888:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11295us (48107us total) trans 4294967713 rc 0/0 00000100:00100000:1.0:1789762828.003712:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 283 00000100:00100000:1.0:1789762828.003721:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301239335030 time 301239461426 state 0x2 ref 0 00000100:00100000:3.0:1789762828.005899:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.005949:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067382016 00000100:00100000:1.0:1789762828.005955:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067382016 00000100:00100000:1.0:1789762828.005988:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301239335030 time 301239461426 state 0x0 ref 1 00000100:00100000:1.0:1789762828.006007:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301289461426 time 301239461426 state 0x1 ref 1 00000100:00100000:1.0:1789762828.042460:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301289461426 time 301289527578 state 0x0 ref 1 00000100:00100000:1.0:1789762828.042483:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067382016:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762828.042528:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 23068672, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0xb007 00010000:00010000:1.0:1789762828.042535:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f1759d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.042548:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:1.0:1789762828.042552:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762828.042556:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f1759d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.042567:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f1759d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762828.053879:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967714 is committed 00002000:00010000:1.0:1789762828.053954:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:1.0:1789762828.053959:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:1.0:1789762828.053969:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f1759d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.053978:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f1759d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.053994:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f1759d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.054000:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.054003:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f1759d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.054010:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f1759d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.054024:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f1759d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.054029:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f1759d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.054088:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067382016:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11612us (48189us total) trans 4294967714 rc 0/0 00000100:00100000:1.0:1789762828.054097:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 284 00000100:00100000:1.0:1789762828.054107:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301289461426 time 301289527578 state 0x2 ref 0 00000100:00100000:3.0:1789762828.056222:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.056260:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067382144 00000100:00100000:1.0:1789762828.056269:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067382144 00000100:00100000:1.0:1789762828.056311:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301289461426 time 301289527578 state 0x0 ref 1 00000100:00100000:1.0:1789762828.056336:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301339527578 time 301289527578 state 0x1 ref 1 00000100:00100000:0.0:1789762828.092531:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301339527578 time 301339600462 state 0x0 ref 1 00000100:00100000:0.0:1789762828.092549:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067382144:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762828.092598:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 24117248, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0xb807 00010000:00010000:0.0:1789762828.092607:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f1759db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.092623:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:0.0:1789762828.092627:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762828.092646:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f1759db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.092662:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f1759db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762828.103439:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967715 is committed 00002000:00010000:1.0:1789762828.103448:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 24117248 -> 25165824 00002000:00010000:1.0:1789762828.103453:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:1.0:1789762828.103463:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f1759db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.103471:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f1759db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.103478:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f1759db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.103483:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.103486:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f1759db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.103493:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f1759db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.103508:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f1759db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.103513:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f1759db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.103578:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067382144:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11032us (47355us total) trans 4294967715 rc 0/0 00000100:00100000:1.0:1789762828.103585:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 285 00000100:00100000:1.0:1789762828.103593:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301339527578 time 301339600462 state 0x2 ref 0 00000100:00100000:2.0:1789762828.105848:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.105892:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067382272 00000100:00100000:1.0:1789762828.105899:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067382272 00000100:00100000:1.0:1789762828.105962:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301339527578 time 301339600462 state 0x0 ref 1 00000100:00100000:1.0:1789762828.105973:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301389600462 time 301339600462 state 0x1 ref 1 00000100:00100000:0.0:1789762828.142644:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301389600462 time 301389702523 state 0x0 ref 1 00000100:00100000:0.0:1789762828.142687:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067382272:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762828.142727:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 25165824, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0xc007 00010000:00010000:0.0:1789762828.142734:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1759e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.142746:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:0.0:1789762828.142749:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762828.142754:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1759e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.142763:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1759e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762828.153688:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967716 is committed 00002000:00010000:1.0:1789762828.153745:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:1.0:1789762828.153750:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:1.0:1789762828.153758:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1759e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.153767:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1759e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.153773:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1759e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.153779:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.153782:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1759e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.153799:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1759e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.153814:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1759e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.153818:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f1759e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.153867:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067382272:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11196us (48018us total) trans 4294967716 rc 0/0 00000100:00100000:1.0:1789762828.153875:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 286 00000100:00100000:1.0:1789762828.153883:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301389600462 time 301389702523 state 0x2 ref 0 00000100:00100000:0.0:1789762828.155991:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.156022:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067382400 00000100:00100000:1.0:1789762828.156029:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067382400 00000100:00100000:1.0:1789762828.156060:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301389600462 time 301389702523 state 0x0 ref 1 00000100:00100000:1.0:1789762828.156070:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301439702523 time 301389702523 state 0x1 ref 1 00000100:00100000:2.0:1789762828.192729:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301439702523 time 301439798276 state 0x0 ref 1 00000100:00100000:2.0:1789762828.192747:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067382400:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762828.192785:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 26214400, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0xc807 00010000:00010000:2.0:1789762828.192792:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.192805:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:2.0:1789762828.192808:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762828.192813:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.192837:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762828.203594:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 26214400 -> 27262976 00000001:00080000:1.0:1789762828.203599:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967717 is committed 00002000:00010000:2.0:1789762828.203601:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:2.0:1789762828.203615:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.203628:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.203653:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.203662:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762828.203668:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.203677:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.203695:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.203701:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762828.203763:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067382400:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11019us (47770us total) trans 4294967717 rc 0/0 00000100:00100000:2.0:1789762828.203772:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 287 00000100:00100000:2.0:1789762828.203783:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301439702523 time 301439798276 state 0x2 ref 0 00000100:00100000:1.0:1789762828.205984:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762828.206032:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067382528 00000100:00100000:0.0:1789762828.206040:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067382528 00000100:00100000:0.0:1789762828.206076:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301439702523 time 301439798276 state 0x0 ref 1 00000100:00100000:0.0:1789762828.206087:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301489798276 time 301439798276 state 0x1 ref 1 00000100:00100000:0.0:1789762828.242788:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301489798276 time 301489859508 state 0x0 ref 1 00000100:00100000:0.0:1789762828.242807:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067382528:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762828.242853:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 27262976, mtime: 0x6aad9d0b, atime: 0x0, ctime: 0x6aad9d0b, blocks: 0xd007 00010000:00010000:0.0:1789762828.242859:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1759f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.242872:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:0.0:1789762828.242875:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762828.242879:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1759f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.242889:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1759f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762828.254225:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967718 is committed 00002000:00010000:1.0:1789762828.254260:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:1.0:1789762828.254263:0:8058:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb mtime from disk: 1789762827 -> 1789762828 00002000:00010000:1.0:1789762828.254271:0:8058:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb ctime from disk: 1789762827 -> 1789762828 00002000:00010000:1.0:1789762828.254272:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 53255 -> 55303 00010000:00010000:1.0:1789762828.254282:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1759f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.254291:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1759f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.254298:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1759f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.254303:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.254306:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1759f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.254313:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1759f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.254326:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1759f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.254332:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f1759f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.254387:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067382528:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11585us (48404us total) trans 4294967718 rc 0/0 00000100:00100000:1.0:1789762828.254396:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 288 00000100:00100000:1.0:1789762828.254410:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301489798276 time 301489859508 state 0x2 ref 0 00000100:00100000:3.0:1789762828.256542:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.256579:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067382656 00000100:00100000:1.0:1789762828.256587:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067382656 00000100:00100000:1.0:1789762828.256620:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301489798276 time 301489859508 state 0x0 ref 1 00000100:00100000:1.0:1789762828.256642:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301539859508 time 301489859508 state 0x1 ref 1 00000100:00100000:2.0:1789762828.292900:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301539859508 time 301539965206 state 0x0 ref 1 00000100:00100000:2.0:1789762828.292957:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067382656:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762828.293014:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 28311552, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0xd807 00010000:00010000:2.0:1789762828.293022:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.293035:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:2.0:1789762828.293039:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762828.293045:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.293056:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762828.303940:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 28311552 -> 29360128 00002000:00010000:1.0:1789762828.303945:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 55303 -> 57351 00010000:00010000:1.0:1789762828.303955:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762828.303959:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967719 is committed 00010000:00010000:1.0:1789762828.303967:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.303975:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.303980:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.303984:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.303990:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.304003:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.304014:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f1759f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.304066:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067382656:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11151us (47524us total) trans 4294967719 rc 0/0 00000100:00100000:1.0:1789762828.304073:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 289 00000100:00100000:1.0:1789762828.304081:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301539859508 time 301539965206 state 0x2 ref 0 00000100:00100000:3.0:1789762828.306272:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762828.306326:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067382784 00000100:00100000:2.0:1789762828.306334:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067382784 00000100:00100000:2.0:1789762828.306371:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301539859508 time 301539965206 state 0x0 ref 1 00000100:00100000:2.0:1789762828.306391:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301589965206 time 301539965206 state 0x1 ref 1 00000100:00100000:1.0:1789762828.342987:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301589965206 time 301590055520 state 0x0 ref 1 00000100:00100000:1.0:1789762828.343007:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067382784:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762828.343042:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 29360128, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0xe007 00010000:00010000:1.0:1789762828.343048:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f1759fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.343060:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:1.0:1789762828.343063:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762828.343068:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f1759fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.343077:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f1759fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762828.354543:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967720 is committed 00002000:00010000:1.0:1789762828.354556:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 29360128 -> 30408704 00002000:00010000:1.0:1789762828.354568:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 57351 -> 59399 00010000:00010000:1.0:1789762828.354577:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f1759fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.354586:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f1759fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.354592:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f1759fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.354598:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.354601:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f1759fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.354607:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f1759fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.354621:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f1759fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.354626:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f1759fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.354698:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067382784:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11695us (48425us total) trans 4294967720 rc 0/0 00000100:00100000:1.0:1789762828.354707:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 290 00000100:00100000:1.0:1789762828.354715:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301589965206 time 301590055520 state 0x2 ref 0 00000100:00100000:3.0:1789762828.356904:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.356960:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067382912 00000100:00100000:1.0:1789762828.356970:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067382912 00000100:00100000:1.0:1789762828.357008:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301589965206 time 301590055520 state 0x0 ref 1 00000100:00100000:1.0:1789762828.357020:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301640055520 time 301590055520 state 0x1 ref 1 00000100:00100000:2.0:1789762828.393095:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301640055520 time 301640164941 state 0x0 ref 1 00000100:00100000:2.0:1789762828.393114:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067382912:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762828.393155:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 30408704, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0xe807 00010000:00010000:2.0:1789762828.393162:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.393174:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:2.0:1789762828.393177:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762828.393182:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.393192:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762828.403897:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 30408704 -> 31457280 00002000:00010000:2.0:1789762828.403902:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 59399 -> 61447 00000001:00080000:3.0:1789762828.403907:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967721 is committed 00010000:00010000:2.0:1789762828.403912:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.403942:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.403959:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.403965:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762828.403968:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.403974:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.403988:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.403994:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762828.404044:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067382912:12345-192.168.203.37@tcp:4:dd.0 Request processed in 10935us (47141us total) trans 4294967721 rc 0/0 00000100:00100000:2.0:1789762828.404051:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 291 00000100:00100000:2.0:1789762828.404059:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301640055520 time 301640164941 state 0x2 ref 0 00000100:00100000:3.0:1789762828.406348:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.406396:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067383040 00000100:00100000:1.0:1789762828.406403:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067383040 00000100:00100000:1.0:1789762828.406438:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301640055520 time 301640164941 state 0x0 ref 1 00000100:00100000:1.0:1789762828.406449:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301690164941 time 301640164941 state 0x1 ref 1 00000100:00100000:0.0:1789762828.443175:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301690164941 time 301690243120 state 0x0 ref 1 00000100:00100000:0.0:1789762828.443195:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067383040:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762828.443239:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 31457280, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0xf007 00010000:00010000:0.0:1789762828.443247:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.443261:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:0.0:1789762828.443265:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762828.443271:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.443284:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762828.454250:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967722 is committed 00002000:00010000:1.0:1789762828.454304:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 31457280 -> 32505856 00002000:00010000:1.0:1789762828.454309:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 61447 -> 63495 00010000:00010000:1.0:1789762828.454319:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.454329:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.454336:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.454341:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.454344:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.454362:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.454377:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.454382:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.454451:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067383040:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11259us (48101us total) trans 4294967722 rc 0/0 00000100:00100000:1.0:1789762828.454460:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 292 00000100:00100000:1.0:1789762828.454469:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301690164941 time 301690243120 state 0x2 ref 0 00000100:00100000:3.0:1789762828.456883:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.456934:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067383168 00000100:00100000:1.0:1789762828.456943:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067383168 00000100:00100000:1.0:1789762828.456980:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301690164941 time 301690243120 state 0x0 ref 1 00000100:00100000:1.0:1789762828.456991:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301740243120 time 301690243120 state 0x1 ref 1 00000100:00100000:0.0:1789762828.493255:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301740243120 time 301740326748 state 0x0 ref 1 00000100:00100000:0.0:1789762828.493273:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067383168:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762828.493311:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 32505856, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0xf807 00010000:00010000:0.0:1789762828.493318:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.493330:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:0.0:1789762828.493334:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762828.493338:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.493349:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762828.503934:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967723 is committed 00002000:00010000:1.0:1789762828.503935:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 32505856 -> 33554432 00002000:00010000:1.0:1789762828.503942:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 63495 -> 65543 00010000:00010000:1.0:1789762828.503951:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.503971:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.503978:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.503983:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.503987:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.503993:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.504007:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.504012:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a22c98/0xdf5a7ac59f175a13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.504065:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067383168:12345-192.168.203.37@tcp:4:dd.0 Request processed in 10796us (47181us total) trans 4294967723 rc 0/0 00000100:00100000:1.0:1789762828.504072:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 293 00000100:00100000:1.0:1789762828.504081:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301740243120 time 301740326748 state 0x2 ref 0 00000100:00100000:3.0:1789762828.506218:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762828.506290:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067383296 00000100:00100000:2.0:1789762828.506299:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067383296 00000100:00100000:2.0:1789762828.506336:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301740243120 time 301740326748 state 0x0 ref 1 00000100:00100000:2.0:1789762828.506356:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301790326748 time 301740326748 state 0x1 ref 1 00000100:00100000:2.0:1789762828.543318:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301790326748 time 301790384490 state 0x0 ref 1 00000100:00100000:2.0:1789762828.543342:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067383296:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762828.543390:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 33554432, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x10007 00010000:00010000:2.0:1789762828.543398:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.543411:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:2.0:1789762828.543415:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762828.543420:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.543431:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762828.554279:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967724 is committed 00002000:00010000:1.0:1789762828.554294:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:1.0:1789762828.554304:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 65543 -> 67591 00010000:00010000:1.0:1789762828.554315:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.554324:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.554330:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.554335:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.554338:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.554345:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.554359:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.554364:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.554420:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067383296:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11085us (48201us total) trans 4294967724 rc 0/0 00000100:00100000:1.0:1789762828.554428:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 294 00000100:00100000:1.0:1789762828.554437:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301790326748 time 301790384490 state 0x2 ref 0 00000100:00100000:3.0:1789762828.556530:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.556564:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067383424 00000100:00100000:1.0:1789762828.556572:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067383424 00000100:00100000:1.0:1789762828.556610:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301790326748 time 301790384490 state 0x0 ref 1 00000100:00100000:1.0:1789762828.556621:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301840384490 time 301790384490 state 0x1 ref 1 00000100:00100000:0.0:1789762828.593402:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301840384490 time 301840468134 state 0x0 ref 1 00000100:00100000:0.0:1789762828.593426:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067383424:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762828.593474:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 34603008, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x10807 00010000:00010000:0.0:1789762828.593482:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.593495:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:0.0:1789762828.593512:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762828.593517:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.593528:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762828.604260:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967725 is committed 00002000:00010000:1.0:1789762828.604276:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:1.0:1789762828.604283:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 67591 -> 69639 00010000:00010000:1.0:1789762828.604296:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.604305:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.604312:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.604317:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.604320:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.604327:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.604341:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.604356:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.604412:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067383424:12345-192.168.203.37@tcp:4:dd.0 Request processed in 10995us (47882us total) trans 4294967725 rc 0/0 00000100:00100000:1.0:1789762828.604421:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 295 00000100:00100000:1.0:1789762828.604430:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301840384490 time 301840468134 state 0x2 ref 0 00000100:00100000:2.0:1789762828.606610:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.606654:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067383552 00000100:00100000:1.0:1789762828.606662:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067383552 00000100:00100000:1.0:1789762828.606698:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301840384490 time 301840468134 state 0x0 ref 1 00000100:00100000:1.0:1789762828.606708:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301890468134 time 301840468134 state 0x1 ref 1 00000100:00100000:1.0:1789762828.643449:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301890468134 time 301890517310 state 0x0 ref 1 00000100:00100000:1.0:1789762828.643470:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067383552:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762828.643513:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 35651584, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x11007 00010000:00010000:1.0:1789762828.643520:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175a28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.643533:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:1.0:1789762828.643536:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762828.643542:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175a28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.643564:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175a28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762828.654812:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967726 is committed 00002000:00010000:1.0:1789762828.654838:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 35651584 -> 36700160 00002000:00010000:1.0:1789762828.654844:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 69639 -> 71687 00010000:00010000:1.0:1789762828.654857:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175a28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.654871:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175a28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.654890:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175a28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.654898:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.654902:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175a28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.654909:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175a28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.654941:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175a28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.654946:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175a28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.655033:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067383552:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11569us (48421us total) trans 4294967726 rc 0/0 00000100:00100000:1.0:1789762828.655042:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 296 00000100:00100000:1.0:1789762828.655051:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301890468134 time 301890517310 state 0x2 ref 0 00000100:00100000:3.0:1789762828.657306:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.657338:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067383680 00000100:00100000:1.0:1789762828.657345:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067383680 00000100:00100000:1.0:1789762828.657387:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301890468134 time 301890517310 state 0x0 ref 1 00000100:00100000:1.0:1789762828.657397:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301940517310 time 301890517310 state 0x1 ref 1 00000100:00100000:2.0:1789762828.693549:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301940517310 time 301940618531 state 0x0 ref 1 00000100:00100000:2.0:1789762828.693567:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067383680:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762828.693605:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 36700160, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x11807 00010000:00010000:2.0:1789762828.693611:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175a2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.693645:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:2.0:1789762828.693649:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762828.693654:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175a2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.693665:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175a2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762828.704015:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967727 is committed 00002000:00010000:1.0:1789762828.704023:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 36700160 -> 37748736 00002000:00010000:1.0:1789762828.704029:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 71687 -> 73737 00010000:00010000:1.0:1789762828.704038:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175a2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.704047:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175a2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.704054:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175a2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.704059:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.704062:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175a2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.704079:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175a2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.704092:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175a2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.704097:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175a2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.704151:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067383680:12345-192.168.203.37@tcp:4:dd.0 Request processed in 10589us (46846us total) trans 4294967727 rc 0/0 00000100:00100000:1.0:1789762828.704159:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 297 00000100:00100000:1.0:1789762828.704168:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301940517310 time 301940618531 state 0x2 ref 0 00000100:00100000:3.0:1789762828.706225:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.706260:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067383808 00000100:00100000:1.0:1789762828.706269:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067383808 00000100:00100000:1.0:1789762828.706304:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301940517310 time 301940618531 state 0x0 ref 1 00000100:00100000:1.0:1789762828.706314:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301990618531 time 301940618531 state 0x1 ref 1 00000100:00100000:2.0:1789762828.743661:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301990618531 time 301990730325 state 0x0 ref 1 00000100:00100000:2.0:1789762828.743680:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067383808:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762828.743721:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 37748736, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x12009 00010000:00010000:2.0:1789762828.743727:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007b374d54/0xdf5a7ac59f175a36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.743740:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=301s v=(0 0) 00010000:00010000:2.0:1789762828.743743:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762828.743748:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007b374d54/0xdf5a7ac59f175a36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.743759:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007b374d54/0xdf5a7ac59f175a36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762828.754584:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967728 is committed 00002000:00010000:1.0:1789762828.754599:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:1.0:1789762828.754604:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 73737 -> 75785 00010000:00010000:1.0:1789762828.754614:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007b374d54/0xdf5a7ac59f175a36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.754649:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007b374d54/0xdf5a7ac59f175a36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.754656:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007b374d54/0xdf5a7ac59f175a36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.754662:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.754665:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007b374d54/0xdf5a7ac59f175a36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.754671:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007b374d54/0xdf5a7ac59f175a36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.754686:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007b374d54/0xdf5a7ac59f175a36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.754692:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007b374d54/0xdf5a7ac59f175a36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.754748:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067383808:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11073us (48522us total) trans 4294967728 rc 0/0 00000100:00100000:1.0:1789762828.754756:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 298 00000100:00100000:1.0:1789762828.754765:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301990618531 time 301990730325 state 0x2 ref 0 00000100:00100000:3.0:1789762828.756882:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.756941:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067383936 00000100:00100000:1.0:1789762828.756955:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067383936 00000100:00100000:1.0:1789762828.756989:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 301990618531 time 301990730325 state 0x0 ref 1 00000100:00100000:1.0:1789762828.757000:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302040730325 time 301990730325 state 0x1 ref 1 00000100:00100000:0.0:1789762828.793775:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302040730325 time 302040838226 state 0x0 ref 1 00000100:00100000:0.0:1789762828.793799:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067383936:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762828.793844:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 38797312, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x12809 00010000:00010000:0.0:1789762828.793851:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f175a3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.793865:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:0.0:1789762828.793868:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762828.793873:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f175a3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.793883:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f175a3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762828.800033:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074675200:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762828.800095:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762828.800127:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074675200 00000100:00100000:1.0:1789762828.800137:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074675200 00000100:00100000:1.0:1789762828.800196:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:1.0:1789762828.800206:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074675200:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789762828.800224:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 335478784 pending 1703936 free 3812622336 avail 3810525184 00000100:00100000:1.0:1789762828.800294:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074675200:12345-0@lo:13:osp-pre-0-0.0 Request processed in 89us (197us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762828.800302:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00080000:1.0:1789762828.800325:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762828.800367:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074675200:0@lo:13:osp-pre-0-0.0 00000004:00080000:3.0:1789762828.801040:0:9022:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 302048120614ns, next at 303048116981ns, rc = 0 00000004:00080000:1.0:1789762828.801074:0:9018:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 302048155271ns, next at 303048151596ns, rc = 0 00000100:00100000:2.0:1789762828.801121:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074675584:0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.801156:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762828.801180:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074675328:0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.801181:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074676224:0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.801208:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074675584 00000100:00100000:3.0:1789762828.801216:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074675584 00000100:00100000:2.0:1789762828.801226:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762828.801232:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762828.801251:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074676608:0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.801277:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074675840:0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.801282:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762828.801301:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074676608 00000100:00100000:3.0:1789762828.801306:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074676608 00000100:00100000:2.0:1789762828.801317:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9245f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074677120:0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.801324:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762828.801351:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762828.801357:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074676352:0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.801382:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:1.0:1789762828.801389:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762828.801391:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff699180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074675456:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762828.801415:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9244a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074676864:0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.801426:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762828.801427:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074676352 00000100:00100000:3.0:1789762828.801432:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074676352 00000100:00100000:1.0:1789762828.801452:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762828.801455:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074675968:0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.801483:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:2.0:1789762828.801483:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762828.801496:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074675968 00000100:00100000:3.0:1789762828.801500:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074675968 00000100:00100000:1.0:1789762828.801502:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff70ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074675712:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762828.801503:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cbbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074676480:0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.801517:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:3.0:1789762828.801526:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074675584:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.801541:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762828.801553:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762828.801564:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9244e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074676992:0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.801580:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074675584:12345-0@lo:400:kworker.0 Request processed in 55us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762828.801587:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:1.0:1789762828.801592:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff70ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074676096:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762828.801600:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762828.801601:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074676480 00000100:00100000:3.0:1789762828.801605:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074676480 00000100:00100000:3.0:1789762828.801645:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:2.0:1789762828.801650:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074675584:0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.801655:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074676608:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.801660:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762828.801693:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9246d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074676736:0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.801696:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074676224 00000100:00100000:2.0:1789762828.801701:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074676224 00000100:00100000:3.0:1789762828.801706:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074676608:12345-0@lo:400:kworker.0 Request processed in 54us (426us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762828.801713:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:3.0:1789762828.801735:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:1.0:1789762828.801739:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762828.801739:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074675840 00000100:00100000:2.0:1789762828.801744:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074675840 00000100:00100000:3.0:1789762828.801746:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9fff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074676352:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.801770:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 278 00000100:00100000:2.0:1789762828.801779:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074676224:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.801790:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9fff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074676352:12345-0@lo:400:kworker.0 Request processed in 47us (401us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762828.801797:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:3.0:1789762828.801811:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074676736 00000100:00100000:3.0:1789762828.801814:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074676736 00000100:00100000:2.0:1789762828.801832:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074676224:12345-0@lo:400:kworker.0 Request processed in 54us (606us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762828.801842:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1789762828.801843:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 278 00000100:00100000:3.0:1789762828.801859:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff684000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074675968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.801865:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 279 00000100:00100000:2.0:1789762828.801875:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9fffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074675840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.801905:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074675328 00000100:00100000:3.0:1789762828.801906:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff684000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074675968:12345-0@lo:400:kworker.0 Request processed in 48us (421us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762828.801914:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074675328 00000100:00100000:3.0:1789762828.801915:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1789762828.801937:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9fffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074675840:12345-0@lo:400:kworker.0 Request processed in 64us (612us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762828.801948:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 279 00000100:00100000:3.0:1789762828.801963:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 416 00000100:00100000:3.0:1789762828.801973:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff684700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074676480:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.801977:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074675456 00000100:00100000:2.0:1789762828.801981:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074675456 00000100:00100000:1.0:1789762828.801991:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:1.0:1789762828.802005:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d157ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074675328:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.802013:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff684700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074676480:12345-0@lo:400:kworker.0 Request processed in 42us (472us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762828.802015:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074675712 00000100:00100000:2.0:1789762828.802021:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074675712 00000100:00100000:3.0:1789762828.802025:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 416 00000100:00100000:3.0:1789762828.802046:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:2.0:1789762828.802053:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074676096 00000100:00100000:3.0:1789762828.802055:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074676736:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.802058:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074676096 00000100:00100000:1.0:1789762828.802064:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d157ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074675328:12345-0@lo:400:kworker.0 Request processed in 61us (831us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762828.802073:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:2.0:1789762828.802084:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 171 00000100:00100000:2.0:1789762828.802094:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff686680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074675456:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762828.802100:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074676736:12345-0@lo:400:kworker.0 Request processed in 47us (367us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762828.802107:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00080000:1.0:1789762828.802112:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00000024:2.0:1789762828.802117:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 13631488 granted 96927744 pending 25559040 free 3858759680 avail 3856662528 00000100:00080000:1.0:1789762828.802120:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789762828.802126:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762828.802148:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074677120 00000100:00100000:3.0:1789762828.802153:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074677120 00000100:00100000:1.0:1789762828.802157:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074676864 00000100:00100000:2.0:1789762828.802161:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff686680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074675456:12345-0@lo:13:osp-pre-1-1.0 Request processed in 68us (734us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762828.802165:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074676864 00000100:00100000:2.0:1789762828.802171:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 171 00000100:00100000:3.0:1789762828.802188:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074676992 00000100:00100000:3.0:1789762828.802193:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074676992 00000100:00100000:2.0:1789762828.802194:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:2.0:1789762828.802205:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074675712:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762828.802211:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 159 00002000:00000024:2.0:1789762828.802215:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 13631488 granted 96927744 pending 25559040 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789762828.802226:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074677120:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.802233:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:3.0:1789762828.802244:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074676864:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.802249:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074675712:12345-0@lo:13:osp-pre-1-0.0 Request processed in 44us (695us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762828.802256:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00100000:2.0:1789762828.802285:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:1.0:1789762828.802286:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074677120:12345-0@lo:400:kworker.0 Request processed in 63us (932us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762828.802294:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074676864:12345-0@lo:400:kworker.0 Request processed in 53us (842us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762828.802296:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074676096:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762828.802302:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 159 00002000:00000024:2.0:1789762828.802310:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 335478784 pending 1703936 free 3812622336 avail 3810525184 00000100:00100000:3.0:1789762828.802310:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:1.0:1789762828.802331:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:1.0:1789762828.802347:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff687480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074676992:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.802347:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074676096:12345-0@lo:13:osp-pre-0-1.0 Request processed in 51us (692us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762828.802361:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000100:00100000:3.0:1789762828.802363:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff699180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074675456:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762828.802379:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074675968:0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.802407:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cbbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074676480:0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.802412:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff687480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074676992:12345-0@lo:400:kworker.0 Request processed in 68us (812us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762828.802423:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9244e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074676992:0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.802437:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000100:00100000:3.0:1789762828.802442:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074676224:0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.802455:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074676608:0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.802468:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9245f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074677120:0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.802600:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff70ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074675712:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762828.802616:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294967698 gen 2 00000004:00080000:1.0:1789762828.802622:0:3427:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37c96a2a00, transno 4294967698 00000100:00100000:1.0:1789762828.802656:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff70ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074676096:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762828.802676:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9246d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074676736:0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.802695:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074675328:0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.802706:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967386 gen 2 00000004:00080000:1.0:1789762828.802711:0:3427:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967386 committed transno 4294967386 00000020:00080000:1.0:1789762828.802716:0:3427:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9c37f5fae300 refcount 2 committed 0 result 0 batchid 8 00000020:00080000:1.0:1789762828.802721:0:3427:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37d19f9c00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9c37f5fae540 00000020:00080000:1.0:1789762828.802724:0:3427:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x2:0x0].8 00000020:00080000:1.0:1789762828.802727:0:3427:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37d19f9a00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9c37d922a800 00000020:00080000:1.0:1789762828.802731:0:3427:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x2:0x0].8 00000100:00100000:1.0:1789762828.802757:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074675840:0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.802771:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074676352:0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.802787:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9244a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074676864:0@lo:400:kworker.0 00000020:00080000:1.0:1789762828.802834:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 8 committed batchid 6 00000020:00080000:1.0:1789762828.802901:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 6 00000001:00080000:1.0:1789762828.806040:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967729 is committed 00002000:00010000:2.0:1789762828.806045:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 38797312 -> 39845888 00002000:00010000:2.0:1789762828.806050:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 75785 -> 77833 00010000:00010000:2.0:1789762828.806060:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f175a3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.806070:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f175a3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.806076:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f175a3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.806081:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762828.806084:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f175a3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.806091:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f175a3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.806105:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f175a3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.806119:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000107139b2/0xdf5a7ac59f175a3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762828.806168:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067383936:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12378us (49288us total) trans 4294967729 rc 0/0 00000100:00100000:2.0:1789762828.806176:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 299 00000100:00100000:2.0:1789762828.806184:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302040730325 time 302040838226 state 0x2 ref 0 00000100:00100000:3.0:1789762828.808556:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.808593:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067384064 00000100:00100000:1.0:1789762828.808599:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067384064 00000100:00100000:1.0:1789762828.808647:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302040730325 time 302040838226 state 0x0 ref 1 00000100:00100000:1.0:1789762828.808659:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302090838226 time 302040838226 state 0x1 ref 1 00000100:00100000:0.0:1789762828.843860:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302090838226 time 302090920395 state 0x0 ref 1 00000100:00100000:0.0:1789762828.843896:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067384064:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762828.843963:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 39845888, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x13009 00010000:00010000:0.0:1789762828.843971:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175a44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.843984:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:0.0:1789762828.843988:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762828.843993:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175a44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.844010:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175a44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762828.855604:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967730 is committed 00002000:00010000:1.0:1789762828.855625:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 39845888 -> 40894464 00002000:00010000:1.0:1789762828.855642:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 77833 -> 79881 00010000:00010000:1.0:1789762828.855653:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175a44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.855666:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175a44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.855677:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175a44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.855685:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.855690:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175a44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.855701:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175a44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.855721:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175a44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.855726:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175a44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.855802:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067384064:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11910us (47244us total) trans 4294967730 rc 0/0 00000100:00100000:1.0:1789762828.855810:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 300 00000100:00100000:1.0:1789762828.855819:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302090838226 time 302090920395 state 0x2 ref 0 00000100:00100000:3.0:1789762828.858145:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.858178:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067384192 00000100:00100000:1.0:1789762828.858193:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067384192 00000100:00100000:1.0:1789762828.858240:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302090838226 time 302090920395 state 0x0 ref 1 00000100:00100000:1.0:1789762828.858255:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302140920395 time 302090920395 state 0x1 ref 1 00000100:00100000:2.0:1789762828.893946:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302140920395 time 302140990158 state 0x0 ref 1 00000100:00100000:2.0:1789762828.893966:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067384192:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762828.894006:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 40894464, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x13809 00010000:00010000:2.0:1789762828.894013:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.894025:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:2.0:1789762828.894029:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762828.894034:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.894044:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762828.905874:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967731 is committed 00002000:00010000:1.0:1789762828.905898:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 40894464 -> 41943040 00002000:00010000:1.0:1789762828.905905:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 79881 -> 81929 00010000:00010000:1.0:1789762828.905936:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.905947:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.905955:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.905960:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.905964:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.905981:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a4b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.905996:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a4b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.906002:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175a4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.906076:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067384192:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12111us (47927us total) trans 4294967731 rc 0/0 00000100:00100000:1.0:1789762828.906087:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 301 00000100:00100000:1.0:1789762828.906100:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302140920395 time 302140990158 state 0x2 ref 0 00000100:00100000:3.0:1789762828.908545:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762828.908597:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067384320 00000100:00100000:2.0:1789762828.908606:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067384320 00000100:00100000:2.0:1789762828.908663:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302140920395 time 302140990158 state 0x0 ref 1 00000100:00100000:2.0:1789762828.908679:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302190990158 time 302140990158 state 0x1 ref 1 00010000:00010000:2.0:1789762828.928018:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762828.944002:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302190990158 time 302191071317 state 0x0 ref 1 00000100:00100000:0.0:1789762828.944023:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067384320:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762828.944076:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 41943040, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x14009 00010000:00010000:0.0:1789762828.944083:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175a52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.944096:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:0.0:1789762828.944099:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762828.944104:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175a52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762828.944129:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175a52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762828.955288:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967732 is committed 00002000:00010000:1.0:1789762828.955309:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 41943040 -> 42991616 00002000:00010000:1.0:1789762828.955316:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 81929 -> 83977 00010000:00010000:1.0:1789762828.955329:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175a52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.955342:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175a52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.955353:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175a52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.955375:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762828.955380:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175a52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.955388:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175a52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.955403:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175a52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762828.955408:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa4962/0xdf5a7ac59f175a52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762828.955472:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067384320:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11454us (46925us total) trans 4294967732 rc 0/0 00000100:00100000:1.0:1789762828.955483:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 302 00000100:00100000:1.0:1789762828.955495:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302190990158 time 302191071317 state 0x2 ref 0 00000100:00100000:3.0:1789762828.957739:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762828.957782:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067384448 00000100:00100000:1.0:1789762828.957791:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067384448 00000100:00100000:1.0:1789762828.957829:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302190990158 time 302191071317 state 0x0 ref 1 00000100:00100000:1.0:1789762828.957840:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302241071317 time 302191071317 state 0x1 ref 1 00000100:00100000:2.0:1789762828.994112:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302241071317 time 302241181206 state 0x0 ref 1 00000100:00100000:2.0:1789762828.994131:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067384448:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762828.994172:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 42991616, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x14809 00010000:00010000:2.0:1789762828.994179:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175a59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.994191:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:2.0:1789762828.994195:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762828.994200:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175a59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762828.994210:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175a59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.005530:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967733 is committed 00002000:00010000:1.0:1789762829.005552:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 42991616 -> 44040192 00002000:00010000:1.0:1789762829.005564:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 83977 -> 86025 00010000:00010000:1.0:1789762829.005577:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175a59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.005588:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175a59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.005595:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175a59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.005600:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.005603:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175a59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.005611:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175a59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.005625:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175a59 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.005639:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175a59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.005701:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067384448:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11572us (47959us total) trans 4294967733 rc 0/0 00000100:00100000:1.0:1789762829.005711:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 303 00000100:00100000:1.0:1789762829.005723:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302241071317 time 302241181206 state 0x2 ref 0 00000100:00100000:3.0:1789762829.007938:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.007981:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067384576 00000100:00100000:1.0:1789762829.007990:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067384576 00000100:00100000:1.0:1789762829.008046:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302241071317 time 302241181206 state 0x0 ref 1 00000100:00080000:2.0:1789762829.008048:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762829.008062:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302291181206 time 302241181206 state 0x1 ref 1 00000100:00100000:1.0:1789762829.044181:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302291181206 time 302291246189 state 0x0 ref 1 00000100:00100000:1.0:1789762829.044204:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067384576:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762829.044255:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 44040192, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x15009 00010000:00010000:1.0:1789762829.044265:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.044282:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:1.0:1789762829.044287:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762829.044294:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.044323:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762829.056038:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 44040192 -> 45088768 00000001:00080000:2.0:1789762829.056043:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967734 is committed 00002000:00010000:1.0:1789762829.056045:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 86025 -> 88073 00010000:00010000:1.0:1789762829.056059:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.056080:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.056087:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.056092:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.056095:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.056102:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.056116:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.056121:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.056188:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067384576:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11990us (48248us total) trans 4294967734 rc 0/0 00000100:00100000:1.0:1789762829.056198:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 304 00000100:00100000:1.0:1789762829.056208:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302291181206 time 302291246189 state 0x2 ref 0 00000100:00100000:3.0:1789762829.058429:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.058467:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067384704 00000100:00100000:1.0:1789762829.058482:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067384704 00000100:00100000:1.0:1789762829.058519:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302291181206 time 302291246189 state 0x0 ref 1 00000100:00100000:1.0:1789762829.058530:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302341246189 time 302291246189 state 0x1 ref 1 00000100:00100000:2.0:1789762829.094282:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302341246189 time 302341344530 state 0x0 ref 1 00000100:00100000:2.0:1789762829.094306:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067384704:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762829.094353:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 45088768, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x15809 00010000:00010000:2.0:1789762829.094360:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.094373:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:2.0:1789762829.094377:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762829.094382:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.094392:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762829.106068:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967735 is committed 00002000:00010000:1.0:1789762829.106083:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 45088768 -> 46137344 00002000:00010000:1.0:1789762829.106088:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 88073 -> 90121 00010000:00010000:1.0:1789762829.106100:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.106114:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.106124:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.106133:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.106138:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.106149:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.106167:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a67 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.106173:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175a67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.106249:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067384704:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11948us (47818us total) trans 4294967735 rc 0/0 00000100:00100000:1.0:1789762829.106259:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 305 00000100:00100000:1.0:1789762829.106269:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302341246189 time 302341344530 state 0x2 ref 0 00000100:00100000:3.0:1789762829.108521:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.108562:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067384832 00000100:00100000:1.0:1789762829.108571:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067384832 00000100:00100000:1.0:1789762829.108612:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302341246189 time 302341344530 state 0x0 ref 1 00000100:00100000:1.0:1789762829.108623:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302391344530 time 302341344530 state 0x1 ref 1 00000100:00100000:1.0:1789762829.144342:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302391344530 time 302391408714 state 0x0 ref 1 00000100:00100000:1.0:1789762829.144363:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067384832:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762829.144406:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 46137344, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x16009 00010000:00010000:1.0:1789762829.144412:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a6e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.144425:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:1.0:1789762829.144429:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762829.144434:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.144445:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762829.155958:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 46137344 -> 47185920 00000001:00080000:3.0:1789762829.155958:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967736 is committed 00002000:00010000:1.0:1789762829.155964:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 90121 -> 92169 00010000:00010000:1.0:1789762829.155974:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.155984:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.155991:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.155997:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.156000:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.156009:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.156027:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.156036:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175a6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.156102:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067384832:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11743us (47580us total) trans 4294967736 rc 0/0 00000100:00100000:1.0:1789762829.156120:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 306 00000100:00100000:1.0:1789762829.156133:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302391344530 time 302391408714 state 0x2 ref 0 00000100:00100000:0.0:1789762829.158264:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.158303:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067384960 00000100:00100000:1.0:1789762829.158313:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067384960 00000100:00100000:1.0:1789762829.158351:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302391344530 time 302391408714 state 0x0 ref 1 00000100:00100000:1.0:1789762829.158362:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302441408714 time 302391408714 state 0x1 ref 1 00000100:00100000:0.0:1789762829.194424:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302441408714 time 302441489918 state 0x0 ref 1 00000100:00100000:0.0:1789762829.194448:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067384960:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762829.194498:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 47185920, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x16809 00010000:00010000:0.0:1789762829.194505:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175a75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.194518:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:0.0:1789762829.194522:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762829.194526:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175a75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.194545:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175a75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762829.205664:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967737 is committed 00002000:00010000:1.0:1789762829.205673:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 47185920 -> 48234496 00002000:00010000:1.0:1789762829.205678:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 92169 -> 94217 00010000:00010000:1.0:1789762829.205690:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175a75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.205699:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175a75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.205706:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175a75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.205711:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.205715:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175a75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.205722:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175a75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.205748:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175a75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.205753:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175a75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.205820:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067384960:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11379us (47554us total) trans 4294967737 rc 0/0 00000100:00100000:1.0:1789762829.205828:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 307 00000100:00100000:1.0:1789762829.205838:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302441408714 time 302441489918 state 0x2 ref 0 00000100:00100000:2.0:1789762829.208176:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.208217:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067385088 00000100:00100000:1.0:1789762829.208228:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067385088 00000100:00100000:1.0:1789762829.208265:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302441408714 time 302441489918 state 0x0 ref 1 00000100:00100000:1.0:1789762829.208280:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302491489918 time 302441489918 state 0x1 ref 1 00000100:00100000:2.0:1789762829.244534:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302491489918 time 302491599037 state 0x0 ref 1 00000100:00100000:2.0:1789762829.244560:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067385088:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762829.244619:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 48234496, mtime: 0x6aad9d0c, atime: 0x0, ctime: 0x6aad9d0c, blocks: 0x17009 00010000:00010000:2.0:1789762829.244658:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175a7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.244672:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:2.0:1789762829.244675:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762829.244680:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175a7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.244691:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175a7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762829.255701:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967738 is committed 00002000:00010000:2.0:1789762829.255714:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 48234496 -> 49283072 00002000:00010000:2.0:1789762829.255720:0:13300:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb mtime from disk: 1789762828 -> 1789762829 00002000:00010000:2.0:1789762829.255724:0:13300:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb ctime from disk: 1789762828 -> 1789762829 00002000:00010000:2.0:1789762829.255728:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 94217 -> 96265 00010000:00010000:2.0:1789762829.255741:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175a7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.255754:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175a7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.255765:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175a7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.255774:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762829.255779:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175a7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.255801:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175a7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.255817:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175a7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.255824:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175a7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762829.255893:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067385088:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11340us (47714us total) trans 4294967738 rc 0/0 00000100:00100000:2.0:1789762829.255911:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 308 00000100:00100000:2.0:1789762829.255948:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302491489918 time 302491599037 state 0x2 ref 0 00000100:00100000:2.0:1789762829.258471:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.258513:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067385216 00000100:00100000:1.0:1789762829.258522:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067385216 00000100:00100000:1.0:1789762829.258564:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302491489918 time 302491599037 state 0x0 ref 1 00000100:00100000:1.0:1789762829.258579:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302541599037 time 302491599037 state 0x1 ref 1 00000100:00100000:0.0:1789762829.294658:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302541599037 time 302541723903 state 0x0 ref 1 00000100:00100000:0.0:1789762829.294682:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067385216:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762829.294732:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 49283072, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x17809 00010000:00010000:0.0:1789762829.294753:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175a83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.294766:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:0.0:1789762829.294770:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762829.294774:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175a83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.294785:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175a83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762829.307012:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967739 is committed 00002000:00010000:1.0:1789762829.307035:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 49283072 -> 50331648 00002000:00010000:1.0:1789762829.307041:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 96265 -> 98313 00010000:00010000:1.0:1789762829.307064:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175a83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.307079:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175a83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.307089:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175a83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.307096:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.307101:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175a83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.307109:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175a83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.307131:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175a83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.307138:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f340a65f/0xdf5a7ac59f175a83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.307226:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067385216:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12550us (48753us total) trans 4294967739 rc 0/0 00000100:00100000:1.0:1789762829.307235:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 309 00000100:00100000:1.0:1789762829.307244:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302541599037 time 302541723903 state 0x2 ref 0 00000100:00100000:1.0:1789762829.309800:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762829.309854:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067385344 00000100:00100000:0.0:1789762829.309863:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067385344 00000100:00100000:0.0:1789762829.309904:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302541599037 time 302541723903 state 0x0 ref 1 00000100:00100000:0.0:1789762829.309915:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302591723903 time 302541723903 state 0x1 ref 1 00000100:00100000:2.0:1789762829.344753:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302591723903 time 302591815236 state 0x0 ref 1 00000100:00100000:2.0:1789762829.344786:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067385344:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762829.344842:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 50331648, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x18009 00010000:00010000:2.0:1789762829.344851:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175a8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.344866:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:2.0:1789762829.344871:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762829.344891:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175a8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.344906:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175a8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.357536:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967740 is committed 00002000:00010000:1.0:1789762829.357553:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 50331648 -> 51380224 00002000:00010000:1.0:1789762829.357558:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 98313 -> 100361 00010000:00010000:1.0:1789762829.357569:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175a8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.357580:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175a8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.357587:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175a8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.357592:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.357596:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175a8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.357602:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175a8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.357617:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175a8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.357622:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175a8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.357713:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067385344:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12938us (47912us total) trans 4294967740 rc 0/0 00000100:00100000:1.0:1789762829.357728:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 310 00000100:00100000:1.0:1789762829.357738:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302591723903 time 302591815236 state 0x2 ref 0 00000100:00100000:2.0:1789762829.360891:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762829.360958:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067385472 00000100:00100000:3.0:1789762829.360969:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067385472 00000100:00100000:3.0:1789762829.361014:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302591723903 time 302591815236 state 0x0 ref 1 00000100:00100000:3.0:1789762829.361026:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302641815236 time 302591815236 state 0x1 ref 1 00000100:00100000:2.0:1789762829.394845:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302641815236 time 302641907121 state 0x0 ref 1 00000100:00100000:2.0:1789762829.394875:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067385472:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762829.394956:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 51380224, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x18809 00010000:00010000:2.0:1789762829.394966:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175a91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.394982:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:2.0:1789762829.394987:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762829.394993:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175a91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.395008:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175a91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.407554:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967741 is committed 00002000:00010000:1.0:1789762829.407599:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 51380224 -> 52428800 00002000:00010000:1.0:1789762829.407605:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 100361 -> 102409 00010000:00010000:1.0:1789762829.407615:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175a91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.407641:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175a91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.407648:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175a91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.407654:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.407657:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175a91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.407665:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175a91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.407680:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175a91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.407686:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175a91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.407769:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067385472:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12904us (46880us total) trans 4294967741 rc 0/0 00000100:00100000:1.0:1789762829.407777:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 311 00000100:00100000:1.0:1789762829.407786:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302641815236 time 302641907121 state 0x2 ref 0 00000100:00100000:0.0:1789762829.410429:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.410467:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067385600 00000100:00100000:1.0:1789762829.410475:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067385600 00000100:00100000:1.0:1789762829.410510:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302641815236 time 302641907121 state 0x0 ref 1 00000100:00100000:1.0:1789762829.410521:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302691907121 time 302641907121 state 0x1 ref 1 00000100:00100000:0.0:1789762829.444948:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302691907121 time 302691994942 state 0x0 ref 1 00000100:00100000:0.0:1789762829.444973:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067385600:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762829.445022:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 52428800, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x19009 00010000:00010000:0.0:1789762829.445032:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.445047:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:0.0:1789762829.445063:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762829.445070:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.445085:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.457961:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967742 is committed 00002000:00010000:1.0:1789762829.457968:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 52428800 -> 53477376 00002000:00010000:1.0:1789762829.457973:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 102409 -> 104457 00010000:00010000:1.0:1789762829.457985:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.457996:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.458002:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.458008:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.458011:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.458028:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.458044:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.458049:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c63f95b/0xdf5a7ac59f175a98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.458121:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067385600:12345-192.168.203.37@tcp:4:dd.0 Request processed in 13155us (47691us total) trans 4294967742 rc 0/0 00000100:00100000:1.0:1789762829.458130:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 312 00000100:00100000:1.0:1789762829.458139:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302691907121 time 302691994942 state 0x2 ref 0 00000100:00100000:2.0:1789762829.461080:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.461115:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067385728 00000100:00100000:1.0:1789762829.461123:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067385728 00000100:00100000:1.0:1789762829.461163:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302691907121 time 302691994942 state 0x0 ref 1 00000100:00100000:1.0:1789762829.461174:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302741994942 time 302691994942 state 0x1 ref 1 00000100:00100000:1.0:1789762829.494991:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302741994942 time 302742056650 state 0x0 ref 1 00000100:00100000:1.0:1789762829.495021:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067385728:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762829.495073:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 53477376, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x19809 00010000:00010000:1.0:1789762829.495092:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175a9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.495106:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:1.0:1789762829.495109:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762829.495115:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175a9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.495125:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175a9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.506401:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967743 is committed 00002000:00010000:1.0:1789762829.506447:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 53477376 -> 54525952 00002000:00010000:1.0:1789762829.506452:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 104457 -> 106505 00010000:00010000:1.0:1789762829.506462:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175a9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.506473:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175a9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.506479:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175a9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.506484:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.506488:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175a9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.506495:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175a9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.506517:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175a9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.506523:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175a9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.506591:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067385728:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11578us (45510us total) trans 4294967743 rc 0/0 00000100:00100000:1.0:1789762829.506598:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 313 00000100:00100000:1.0:1789762829.506608:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302741994942 time 302742056650 state 0x2 ref 0 00000100:00100000:0.0:1789762829.509206:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.509243:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067385856 00000100:00100000:1.0:1789762829.509259:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067385856 00000100:00100000:1.0:1789762829.509296:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302741994942 time 302742056650 state 0x0 ref 1 00000100:00100000:1.0:1789762829.509307:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302792056650 time 302742056650 state 0x1 ref 1 00000100:00100000:2.0:1789762829.545092:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302792056650 time 302792157539 state 0x0 ref 1 00000100:00100000:2.0:1789762829.545115:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067385856:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762829.545164:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 54525952, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x1a009 00010000:00010000:2.0:1789762829.545173:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175aa6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.545188:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:2.0:1789762829.545193:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762829.545200:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175aa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.545213:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175aa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.557304:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967744 is committed 00002000:00010000:1.0:1789762829.557330:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 54525952 -> 55574528 00002000:00010000:1.0:1789762829.557336:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 106505 -> 108553 00010000:00010000:1.0:1789762829.557348:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175aa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.557358:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175aa6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.557376:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175aa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.557381:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.557385:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175aa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.557392:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175aa6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.557407:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175aa6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.557413:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175aa6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.557489:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067385856:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12378us (48282us total) trans 4294967744 rc 0/0 00000100:00100000:1.0:1789762829.557497:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 314 00000100:00100000:1.0:1789762829.557508:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302792056650 time 302792157539 state 0x2 ref 0 00000100:00100000:0.0:1789762829.560234:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.560282:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067385984 00000100:00100000:1.0:1789762829.560291:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067385984 00000100:00100000:1.0:1789762829.560331:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302792056650 time 302792157539 state 0x0 ref 1 00000100:00100000:1.0:1789762829.560353:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302842157539 time 302792157539 state 0x1 ref 1 00000100:00100000:3.0:1789762829.595168:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302842157539 time 302842232552 state 0x0 ref 1 00000100:00100000:3.0:1789762829.595209:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067385984:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762829.595258:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 55574528, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x1a809 00010000:00010000:3.0:1789762829.595265:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175aad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.595278:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:3.0:1789762829.595282:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762829.595287:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175aad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.595298:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175aad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.606837:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967745 is committed 00002000:00010000:1.0:1789762829.606854:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 55574528 -> 56623104 00002000:00010000:1.0:1789762829.606859:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 108553 -> 110601 00010000:00010000:1.0:1789762829.606871:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175aad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.606882:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175aad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.606889:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175aad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.606894:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d92000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.606897:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175aad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.606904:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175aad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.606935:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175aad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.606941:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fa22910/0xdf5a7ac59f175aad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.607019:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067385984:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11815us (46786us total) trans 4294967745 rc 0/0 00000100:00100000:1.0:1789762829.607029:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 315 00000100:00100000:1.0:1789762829.607039:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302842157539 time 302842232552 state 0x2 ref 0 00000100:00100000:2.0:1789762829.609844:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.609888:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067386112 00000100:00100000:1.0:1789762829.609897:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067386112 00000100:00100000:1.0:1789762829.609953:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302842157539 time 302842232552 state 0x0 ref 1 00000100:00100000:1.0:1789762829.609964:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302892232552 time 302842232552 state 0x1 ref 1 00000100:00100000:2.0:1789762829.645275:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302892232552 time 302892341059 state 0x0 ref 1 00000100:00100000:2.0:1789762829.645300:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067386112:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762829.645349:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 56623104, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x1b009 00010000:00010000:2.0:1789762829.645356:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175ab4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.645386:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:2.0:1789762829.645389:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762829.645394:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175ab4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.645405:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175ab4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.656733:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967746 is committed 00002000:00010000:1.0:1789762829.656737:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 56623104 -> 57671680 00002000:00010000:1.0:1789762829.656742:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 110601 -> 112649 00010000:00010000:1.0:1789762829.656753:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175ab4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.656764:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175ab4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.656771:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175ab4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.656777:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.656780:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175ab4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.656797:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175ab4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.656813:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175ab4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.656818:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175ab4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.656884:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067386112:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11591us (47037us total) trans 4294967746 rc 0/0 00000100:00100000:1.0:1789762829.656893:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 316 00000100:00100000:1.0:1789762829.656903:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302892232552 time 302892341059 state 0x2 ref 0 00000100:00100000:0.0:1789762829.659186:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.659266:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067386240 00000100:00100000:1.0:1789762829.659275:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067386240 00000100:00100000:1.0:1789762829.659316:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302892232552 time 302892341059 state 0x0 ref 1 00000100:00100000:1.0:1789762829.659327:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302942341059 time 302892341059 state 0x1 ref 1 00000100:00100000:3.0:1789762829.695361:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302942341059 time 302942425523 state 0x0 ref 1 00000100:00100000:3.0:1789762829.695386:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067386240:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762829.695436:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 57671680, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x1b809 00010000:00010000:3.0:1789762829.695443:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175abb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.695456:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:3.0:1789762829.695460:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762829.695464:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175abb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.695490:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175abb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.706881:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967747 is committed 00002000:00010000:3.0:1789762829.706896:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 57671680 -> 58720256 00002000:00010000:3.0:1789762829.706903:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 112649 -> 114697 00010000:00010000:3.0:1789762829.706940:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175abb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.706955:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175abb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.706967:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175abb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.706976:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762829.706981:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175abb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.706991:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175abb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.707010:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175abb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.707019:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175abb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762829.707111:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067386240:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11731us (47923us total) trans 4294967747 rc 0/0 00000100:00100000:3.0:1789762829.707124:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 317 00000100:00100000:3.0:1789762829.707137:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302942341059 time 302942425523 state 0x2 ref 0 00000100:00100000:1.0:1789762829.709546:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762829.709585:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067386368 00000100:00100000:3.0:1789762829.709593:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067386368 00000100:00100000:3.0:1789762829.709645:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302942341059 time 302942425523 state 0x0 ref 1 00000100:00100000:3.0:1789762829.709656:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302992425523 time 302942425523 state 0x1 ref 1 00000100:00100000:2.0:1789762829.745473:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302992425523 time 302992537721 state 0x0 ref 1 00000100:00100000:2.0:1789762829.745496:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067386368:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762829.745544:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 58720256, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x1c009 00010000:00010000:2.0:1789762829.745551:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175ac2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.745565:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=302s v=(0 0) 00010000:00010000:2.0:1789762829.745570:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762829.745577:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175ac2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.745593:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175ac2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.757403:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967748 is committed 00002000:00010000:1.0:1789762829.757415:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 58720256 -> 59768832 00002000:00010000:1.0:1789762829.757421:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 114697 -> 116745 00010000:00010000:1.0:1789762829.757433:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175ac2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.757454:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175ac2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.757460:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175ac2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.757465:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.757469:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175ac2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.757476:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175ac2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.757492:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175ac2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.757497:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f175ac2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.757566:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067386368:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12076us (48017us total) trans 4294967748 rc 0/0 00000100:00100000:1.0:1789762829.757575:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 318 00000100:00100000:1.0:1789762829.757584:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302992425523 time 302992537721 state 0x2 ref 0 00000100:00100000:0.0:1789762829.759899:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.759972:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067386496 00000100:00100000:1.0:1789762829.759987:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067386496 00000100:00100000:1.0:1789762829.760023:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 302992425523 time 302992537721 state 0x0 ref 1 00000100:00100000:1.0:1789762829.760035:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303042537721 time 302992537721 state 0x1 ref 1 00000100:00100000:0.0:1789762829.795576:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303042537721 time 303042639978 state 0x0 ref 1 00000100:00100000:0.0:1789762829.795604:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067386496:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762829.795669:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 59768832, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x1c809 00010000:00010000:0.0:1789762829.795679:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175ac9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.795695:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:0.0:1789762829.795700:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762829.795707:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175ac9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.795722:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175ac9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.807421:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967749 is committed 00002000:00010000:1.0:1789762829.807443:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 59768832 -> 60817408 00002000:00010000:1.0:1789762829.807448:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 116745 -> 118793 00010000:00010000:1.0:1789762829.807458:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175ac9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.807468:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175ac9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.807475:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175ac9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.807479:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.807483:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175ac9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.807490:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175ac9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.807504:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175ac9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.807509:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175ac9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.807588:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067386496:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11993us (47688us total) trans 4294967749 rc 0/0 00000100:00100000:1.0:1789762829.807597:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 319 00000100:00100000:1.0:1789762829.807606:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303042537721 time 303042639978 state 0x2 ref 0 00000100:00100000:0.0:1789762829.810234:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762829.810281:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067386624 00000100:00100000:3.0:1789762829.810291:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067386624 00000100:00100000:3.0:1789762829.810331:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303042537721 time 303042639978 state 0x0 ref 1 00000100:00100000:3.0:1789762829.810342:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303092639978 time 303042639978 state 0x1 ref 1 00000100:00100000:1.0:1789762829.845712:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303092639978 time 303092777928 state 0x0 ref 1 00000100:00100000:1.0:1789762829.845748:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067386624:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762829.845800:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 60817408, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x1d009 00010000:00010000:1.0:1789762829.845807:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ad0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.845820:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:1.0:1789762829.845824:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762829.845828:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ad0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.845840:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ad0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.857714:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967750 is committed 00002000:00010000:1.0:1789762829.857723:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 60817408 -> 61865984 00002000:00010000:1.0:1789762829.857729:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 118793 -> 120841 00010000:00010000:1.0:1789762829.857742:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ad0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.857763:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ad0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.857770:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ad0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.857775:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.857778:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ad0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.857784:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ad0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.857800:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ad0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.857806:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ad0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.857874:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067386624:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12145us (47639us total) trans 4294967750 rc 0/0 00000100:00100000:1.0:1789762829.857882:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 320 00000100:00100000:1.0:1789762829.857892:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303092639978 time 303092777928 state 0x2 ref 0 00000100:00100000:2.0:1789762829.860472:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.860540:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067386752 00000100:00100000:1.0:1789762829.860556:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067386752 00000100:00100000:1.0:1789762829.860594:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303092639978 time 303092777928 state 0x0 ref 1 00000100:00100000:1.0:1789762829.860605:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303142777928 time 303092777928 state 0x1 ref 1 00000100:00100000:2.0:1789762829.895806:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303142777928 time 303142871775 state 0x0 ref 1 00000100:00100000:2.0:1789762829.895829:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067386752:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762829.895875:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 61865984, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x1d809 00010000:00010000:2.0:1789762829.895883:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175ad7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.895904:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:2.0:1789762829.895908:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762829.895913:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175ad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.895952:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175ad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762829.907307:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967751 is committed 00002000:00010000:1.0:1789762829.907362:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 61865984 -> 62914560 00002000:00010000:1.0:1789762829.907366:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 120841 -> 122889 00010000:00010000:1.0:1789762829.907376:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175ad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.907386:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175ad7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.907393:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175ad7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.907398:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.907401:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175ad7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.907408:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175ad7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.907422:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175ad7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.907427:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175ad7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.907506:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067386752:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11683us (47033us total) trans 4294967751 rc 0/0 00000100:00100000:1.0:1789762829.907514:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 321 00000100:00100000:1.0:1789762829.907524:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303142777928 time 303142871775 state 0x2 ref 0 00000100:00100000:2.0:1789762829.910049:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762829.910084:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067386880 00000100:00100000:1.0:1789762829.910095:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067386880 00000100:00100000:1.0:1789762829.910133:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303142777928 time 303142871775 state 0x0 ref 1 00000100:00100000:1.0:1789762829.910144:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303192871775 time 303142871775 state 0x1 ref 1 00000100:00100000:1.0:1789762829.945898:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303192871775 time 303192962774 state 0x0 ref 1 00000100:00100000:1.0:1789762829.945954:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067386880:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762829.946004:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 62914560, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x1e009 00010000:00010000:1.0:1789762829.946011:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ade lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.946024:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:1.0:1789762829.946028:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762829.946032:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ade lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.946043:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ade lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.952004:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762829.958250:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967752 is committed 00002000:00010000:1.0:1789762829.958332:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 62914560 -> 63963136 00002000:00010000:1.0:1789762829.958339:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 122889 -> 124937 00010000:00010000:1.0:1789762829.958352:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ade lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.958367:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ade lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.958377:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ade lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.958386:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762829.958391:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ade lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.958401:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ade lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.958419:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ade lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.958428:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175ade lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.958510:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067386880:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12593us (48457us total) trans 4294967752 rc 0/0 00000100:00100000:1.0:1789762829.958531:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 322 00000100:00100000:1.0:1789762829.958543:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303192871775 time 303192962774 state 0x2 ref 0 00000100:00100000:1.0:1789762829.961163:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762829.961220:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067387008 00000100:00100000:2.0:1789762829.961230:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067387008 00000100:00100000:2.0:1789762829.961275:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303192871775 time 303192962774 state 0x0 ref 1 00000100:00100000:2.0:1789762829.961297:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303242962774 time 303192962774 state 0x1 ref 1 00000100:00100000:2.0:1789762829.995984:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303242962774 time 303243044232 state 0x0 ref 1 00000100:00100000:2.0:1789762829.996017:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067387008:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762829.996074:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 63963136, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x1e809 00010000:00010000:2.0:1789762829.996084:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f175ae5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.996101:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:2.0:1789762829.996106:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762829.996113:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f175ae5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.996129:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f175ae5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762830.009331:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967753 is committed 00002000:00010000:1.0:1789762830.009349:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 63963136 -> 65011712 00002000:00010000:1.0:1789762830.009354:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 124937 -> 126985 00010000:00010000:1.0:1789762830.009366:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f175ae5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.009377:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f175ae5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.009383:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f175ae5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.009388:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762830.009392:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f175ae5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.009399:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f175ae5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.009416:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f175ae5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.009423:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f175ae5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762830.009510:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067387008:12345-192.168.203.37@tcp:4:dd.0 Request processed in 13504us (48345us total) trans 4294967753 rc 0/0 00000100:00100000:1.0:1789762830.009519:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 323 00000100:00100000:1.0:1789762830.009528:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303242962774 time 303243044232 state 0x2 ref 0 00000100:00100000:0.0:1789762830.011948:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762830.011986:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067387136 00000100:00100000:1.0:1789762830.011994:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067387136 00000100:00100000:1.0:1789762830.012036:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303242962774 time 303243044232 state 0x0 ref 1 00000100:00100000:1.0:1789762830.012051:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303293044232 time 303243044232 state 0x1 ref 1 00000100:00100000:3.0:1789762830.046079:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303293044232 time 303293140386 state 0x0 ref 1 00000100:00100000:3.0:1789762830.046112:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067387136:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762830.046178:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 65011712, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x1f009 00010000:00010000:3.0:1789762830.046187:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175aec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.046219:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:3.0:1789762830.046223:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762830.046227:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175aec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.046238:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175aec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762830.059358:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967754 is committed 00002000:00010000:1.0:1789762830.059387:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 65011712 -> 66060288 00002000:00010000:1.0:1789762830.059394:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 126985 -> 129033 00010000:00010000:1.0:1789762830.059408:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175aec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.059423:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175aec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.059434:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175aec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.059442:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762830.059448:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175aec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.059473:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175aec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.059494:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175aec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.059503:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175aec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762830.059588:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067387136:12345-192.168.203.37@tcp:4:dd.0 Request processed in 13487us (47640us total) trans 4294967754 rc 0/0 00000100:00100000:1.0:1789762830.059600:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 324 00000100:00100000:1.0:1789762830.059614:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303293044232 time 303293140386 state 0x2 ref 0 00000100:00100000:2.0:1789762830.062362:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762830.062411:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067387264 00000100:00100000:1.0:1789762830.062420:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067387264 00000100:00100000:1.0:1789762830.062469:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303293044232 time 303293140386 state 0x0 ref 1 00000100:00100000:1.0:1789762830.062485:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303343140386 time 303293140386 state 0x1 ref 1 00000100:00100000:2.0:1789762830.096193:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303343140386 time 303343254234 state 0x0 ref 1 00000100:00100000:2.0:1789762830.096226:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067387264:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762830.096277:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 66060288, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x1f809 00010000:00010000:2.0:1789762830.096297:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175af3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.096311:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:2.0:1789762830.096314:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762830.096319:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175af3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.096330:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175af3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762830.108684:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967755 is committed 00002000:00010000:1.0:1789762830.108700:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 66060288 -> 67108864 00002000:00010000:1.0:1789762830.108706:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 129033 -> 131081 00010000:00010000:1.0:1789762830.108719:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175af3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.108739:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175af3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.108746:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175af3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.108751:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762830.108754:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175af3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.108761:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175af3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.108775:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175af3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.108780:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f175af3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762830.108844:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067387264:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12631us (46482us total) trans 4294967755 rc 0/0 00000100:00100000:1.0:1789762830.108853:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 325 00000100:00100000:1.0:1789762830.108862:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303343140386 time 303343254234 state 0x2 ref 0 00000100:00100000:0.0:1789762830.111393:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762830.111440:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067387392 00000100:00100000:1.0:1789762830.111455:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067387392 00000100:00100000:1.0:1789762830.111498:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303343140386 time 303343254234 state 0x0 ref 1 00000100:00100000:1.0:1789762830.111512:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303393254234 time 303343254234 state 0x1 ref 1 00000100:00100000:0.0:1789762830.146300:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303393254234 time 303393366564 state 0x0 ref 1 00000100:00100000:0.0:1789762830.146324:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067387392:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762830.146372:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 67108864, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x20009 00010000:00010000:0.0:1789762830.146379:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175afa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.146392:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:0.0:1789762830.146396:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762830.146400:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175afa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.146411:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175afa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762830.158271:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967756 is committed 00002000:00010000:1.0:1789762830.158285:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 67108864 -> 68157440 00002000:00010000:1.0:1789762830.158291:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 131081 -> 133129 00010000:00010000:1.0:1789762830.158300:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175afa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.158310:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175afa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.158317:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175afa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.158323:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762830.158326:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175afa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.158332:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175afa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.158346:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175afa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.158352:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175afa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762830.158422:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067387392:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12106us (47030us total) trans 4294967756 rc 0/0 00000100:00100000:1.0:1789762830.158431:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 326 00000100:00100000:1.0:1789762830.158440:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303393254234 time 303393366564 state 0x2 ref 0 00000100:00100000:2.0:1789762830.161042:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762830.161080:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067387520 00000100:00100000:1.0:1789762830.161090:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067387520 00000100:00100000:1.0:1789762830.161135:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303393254234 time 303393366564 state 0x0 ref 1 00000100:00100000:1.0:1789762830.161150:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303443366564 time 303393366564 state 0x1 ref 1 00000100:00100000:2.0:1789762830.196414:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303443366564 time 303443477634 state 0x0 ref 1 00000100:00100000:2.0:1789762830.196442:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067387520:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762830.196493:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 68157440, mtime: 0x6aad9d0d, atime: 0x0, ctime: 0x6aad9d0d, blocks: 0x20809 00010000:00010000:2.0:1789762830.196502:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175b01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.196518:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:2.0:1789762830.196523:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762830.196530:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175b01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.196545:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175b01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762830.207895:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967757 is committed 00002000:00010000:1.0:1789762830.207900:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 68157440 -> 69206016 00002000:00010000:1.0:1789762830.207906:0:8059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb mtime from disk: 1789762829 -> 1789762830 00002000:00010000:1.0:1789762830.207910:0:8059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb ctime from disk: 1789762829 -> 1789762830 00002000:00010000:1.0:1789762830.207913:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 133129 -> 135177 00010000:00010000:1.0:1789762830.207949:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175b01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.207963:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175b01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.207973:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175b01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.207981:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762830.207990:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175b01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.207997:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175b01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.208012:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175b01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.208017:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f175b01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762830.208085:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067387520:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11650us (47040us total) trans 4294967757 rc 0/0 00000100:00100000:1.0:1789762830.208096:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 327 00000100:00100000:1.0:1789762830.208107:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303443366564 time 303443477634 state 0x2 ref 0 00000100:00100000:0.0:1789762830.210588:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762830.210642:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067387648 00000100:00100000:1.0:1789762830.210651:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067387648 00000100:00100000:1.0:1789762830.210694:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303443366564 time 303443477634 state 0x0 ref 1 00000100:00100000:1.0:1789762830.210710:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303493477634 time 303443477634 state 0x1 ref 1 00000100:00100000:2.0:1789762830.246505:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303493477634 time 303493566388 state 0x0 ref 1 00000100:00100000:2.0:1789762830.246534:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067387648:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762830.246585:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 69206016, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x21009 00010000:00010000:2.0:1789762830.246595:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.246611:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:2.0:1789762830.246630:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762830.246638:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.246653:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762830.257973:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967758 is committed 00002000:00010000:1.0:1789762830.258036:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 69206016 -> 70254592 00002000:00010000:1.0:1789762830.258041:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 135177 -> 137227 00010000:00010000:1.0:1789762830.258051:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.258073:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.258083:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.258092:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762830.258097:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.258108:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.258127:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.258134:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762830.258211:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067387648:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11686us (47621us total) trans 4294967758 rc 0/0 00000100:00100000:1.0:1789762830.258221:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 328 00000100:00100000:1.0:1789762830.258231:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303493477634 time 303493566388 state 0x2 ref 0 00000100:00100000:0.0:1789762830.260799:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762830.260841:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067387776 00000100:00100000:1.0:1789762830.260857:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067387776 00000100:00100000:1.0:1789762830.260897:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303493477634 time 303493566388 state 0x0 ref 1 00000100:00100000:1.0:1789762830.260913:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303543566388 time 303493566388 state 0x1 ref 1 00000100:00100000:3.0:1789762830.296586:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303543566388 time 303543653092 state 0x0 ref 1 00000100:00100000:3.0:1789762830.296608:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067387776:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762830.296670:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 70254592, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2180b 00010000:00010000:3.0:1789762830.296677:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175b0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.296689:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:3.0:1789762830.296693:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762830.296698:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175b0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.296708:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175b0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762830.307703:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967759 is committed 00002000:00010000:1.0:1789762830.307740:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 70254592 -> 71303168 00002000:00010000:1.0:1789762830.307747:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 137227 -> 139275 00010000:00010000:1.0:1789762830.307758:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175b0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.307773:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175b0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.307782:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175b0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.307791:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762830.307796:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175b0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.307807:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175b0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.307827:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175b0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.307837:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79a19ad/0xdf5a7ac59f175b0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762830.307953:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067387776:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11318us (47120us total) trans 4294967759 rc 0/0 00000100:00100000:1.0:1789762830.307965:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 329 00000100:00100000:1.0:1789762830.307978:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303543566388 time 303543653092 state 0x2 ref 0 00000100:00100000:0.0:1789762830.310389:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762830.310431:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067387904 00000100:00100000:1.0:1789762830.310439:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067387904 00000100:00100000:1.0:1789762830.310475:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303543566388 time 303543653092 state 0x0 ref 1 00000100:00100000:1.0:1789762830.310486:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303593653092 time 303543653092 state 0x1 ref 1 00000100:00100000:0.0:1789762830.346650:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303593653092 time 303593717555 state 0x0 ref 1 00000100:00100000:0.0:1789762830.346678:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067387904:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762830.346793:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 71303168, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2200b 00010000:00010000:0.0:1789762830.346802:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175b16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.346815:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:0.0:1789762830.346840:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762830.346846:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175b16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.346860:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175b16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762830.356896:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967379 is committed 00000001:00080000:0.0:1789762830.356965:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967380 is committed 00000001:00080000:1.0:1789762830.361168:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967760 is committed 00002000:00010000:0.0:1789762830.361193:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 71303168 -> 72351744 00002000:00010000:0.0:1789762830.361200:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 139275 -> 141323 00010000:00010000:0.0:1789762830.361214:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175b16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.361228:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175b16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.361239:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175b16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.361246:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762830.361250:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175b16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.361264:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175b16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.361280:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175b16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.361285:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eab1955/0xdf5a7ac59f175b16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762830.361352:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067387904:12345-192.168.203.37@tcp:4:dd.0 Request processed in 14682us (50962us total) trans 4294967760 rc 0/0 00000100:00100000:0.0:1789762830.361360:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 330 00000100:00100000:0.0:1789762830.361370:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303593653092 time 303593717555 state 0x2 ref 0 00000100:00100000:3.0:1789762830.363963:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762830.364007:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067388032 00000100:00100000:1.0:1789762830.364016:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067388032 00000100:00100000:1.0:1789762830.364056:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303593653092 time 303593717555 state 0x0 ref 1 00000100:00100000:1.0:1789762830.364070:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303643717555 time 303593717555 state 0x1 ref 1 00000100:00100000:3.0:1789762830.396757:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303643717555 time 303643818914 state 0x0 ref 1 00000100:00100000:3.0:1789762830.396802:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067388032:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762830.396855:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 72351744, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2280b 00010000:00010000:3.0:1789762830.396863:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175b1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.396876:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:3.0:1789762830.396880:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762830.396885:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175b1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.396896:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175b1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762830.409509:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967761 is committed 00002000:00010000:0.0:1789762830.409548:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 72351744 -> 73400320 00002000:00010000:0.0:1789762830.409554:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 141323 -> 143371 00010000:00010000:0.0:1789762830.409566:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175b1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.409579:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175b1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.409589:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175b1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.409610:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d92a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762830.409627:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175b1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.409639:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175b1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.409661:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175b1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.409669:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e63a7a0/0xdf5a7ac59f175b1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762830.409748:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067388032:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12953us (45782us total) trans 4294967761 rc 0/0 00000100:00100000:0.0:1789762830.409760:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 331 00000100:00100000:0.0:1789762830.409773:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303643717555 time 303643818914 state 0x2 ref 0 00000100:00100000:2.0:1789762830.412334:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762830.412383:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067388160 00000100:00100000:1.0:1789762830.412391:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067388160 00000100:00100000:1.0:1789762830.412431:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303643717555 time 303643818914 state 0x0 ref 1 00000100:00100000:1.0:1789762830.412442:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303693818914 time 303643818914 state 0x1 ref 1 00000100:00100000:2.0:1789762830.446830:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303693818914 time 303693896716 state 0x0 ref 1 00000100:00100000:2.0:1789762830.446853:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067388160:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762830.446902:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 73400320, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2300b 00010000:00010000:2.0:1789762830.446909:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175b24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.446960:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:2.0:1789762830.446964:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762830.446969:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175b24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.446981:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175b24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762830.458147:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 73400320 -> 74448896 00000001:00080000:3.0:1789762830.458150:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967762 is committed 00002000:00010000:2.0:1789762830.458154:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 143371 -> 145419 00010000:00010000:2.0:1789762830.458165:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175b24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.458175:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175b24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.458182:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175b24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.458188:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762830.458191:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175b24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.458197:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175b24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.458221:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175b24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.458227:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175b24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762830.458288:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067388160:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11442us (45954us total) trans 4294967762 rc 0/0 00000100:00100000:2.0:1789762830.458297:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 332 00000100:00100000:2.0:1789762830.458306:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303693818914 time 303693896716 state 0x2 ref 0 00000100:00100000:3.0:1789762830.460784:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762830.460827:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067388288 00000100:00100000:0.0:1789762830.460835:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067388288 00000100:00100000:0.0:1789762830.460873:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303693818914 time 303693896716 state 0x0 ref 1 00000100:00100000:0.0:1789762830.460883:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303743896716 time 303693896716 state 0x1 ref 1 00000100:00100000:0.0:1789762830.496868:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303743896716 time 303743936945 state 0x0 ref 1 00000100:00100000:0.0:1789762830.496898:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067388288:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762830.496957:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 74448896, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2380b 00010000:00010000:0.0:1789762830.496965:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175b2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.496977:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:0.0:1789762830.496981:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762830.496986:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175b2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.496996:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175b2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762830.507894:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967763 is committed 00002000:00010000:0.0:1789762830.507976:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:0.0:1789762830.507981:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 145419 -> 147467 00010000:00010000:0.0:1789762830.507993:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175b2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.508005:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175b2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.508015:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175b2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.508023:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762830.508035:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175b2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.508044:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175b2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.508064:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175b2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.508074:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175b2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762830.508148:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067388288:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11251us (47362us total) trans 4294967763 rc 0/0 00000100:00100000:0.0:1789762830.508158:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 333 00000100:00100000:0.0:1789762830.508171:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303743896716 time 303743936945 state 0x2 ref 0 00000100:00100000:3.0:1789762830.510318:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762830.510363:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067388416 00000100:00100000:0.0:1789762830.510371:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067388416 00000100:00100000:0.0:1789762830.510415:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303743896716 time 303743936945 state 0x0 ref 1 00000100:00100000:0.0:1789762830.510430:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303793936945 time 303743936945 state 0x1 ref 1 00000100:00100000:2.0:1789762830.546954:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303793936945 time 303794023568 state 0x0 ref 1 00000100:00100000:2.0:1789762830.546975:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067388416:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762830.547017:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 75497472, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2400b 00010000:00010000:2.0:1789762830.547026:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175b32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.547042:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:2.0:1789762830.547047:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762830.547054:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175b32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.547077:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175b32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762830.557994:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967764 is committed 00002000:00010000:3.0:1789762830.557995:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 75497472 -> 76546048 00002000:00010000:3.0:1789762830.558001:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 147467 -> 149515 00010000:00010000:3.0:1789762830.558011:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175b32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.558021:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175b32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.558029:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175b32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.558044:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762830.558049:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175b32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.558059:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175b32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.558079:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175b32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.558085:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f175b32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762830.558140:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067388416:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11170us (47822us total) trans 4294967764 rc 0/0 00000100:00100000:3.0:1789762830.558148:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 334 00000100:00100000:3.0:1789762830.558157:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303793936945 time 303794023568 state 0x2 ref 0 00000100:00100000:3.0:1789762830.560262:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762830.560312:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067388544 00000100:00100000:2.0:1789762830.560321:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067388544 00000100:00100000:2.0:1789762830.560359:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303793936945 time 303794023568 state 0x0 ref 1 00000100:00100000:2.0:1789762830.560370:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303844023568 time 303794023568 state 0x1 ref 1 00000100:00100000:1.0:1789762830.597038:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303844023568 time 303844105465 state 0x0 ref 1 00000100:00100000:1.0:1789762830.597059:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067388544:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762830.597112:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 76546048, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2480b 00010000:00010000:1.0:1789762830.597120:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ac9f7e4/0xdf5a7ac59f175b39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.597134:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:1.0:1789762830.597137:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762830.597142:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ac9f7e4/0xdf5a7ac59f175b39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.597153:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ac9f7e4/0xdf5a7ac59f175b39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762830.607804:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967765 is committed 00002000:00010000:0.0:1789762830.607852:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 76546048 -> 77594624 00002000:00010000:0.0:1789762830.607857:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 149515 -> 151563 00010000:00010000:0.0:1789762830.607867:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ac9f7e4/0xdf5a7ac59f175b39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.607884:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ac9f7e4/0xdf5a7ac59f175b39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.607890:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ac9f7e4/0xdf5a7ac59f175b39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.607896:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762830.607899:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ac9f7e4/0xdf5a7ac59f175b39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.607905:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ac9f7e4/0xdf5a7ac59f175b39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.607935:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ac9f7e4/0xdf5a7ac59f175b39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.607941:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ac9f7e4/0xdf5a7ac59f175b39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762830.608002:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067388544:12345-192.168.203.37@tcp:4:dd.0 Request processed in 10949us (47739us total) trans 4294967765 rc 0/0 00000100:00100000:0.0:1789762830.608010:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 335 00000100:00100000:0.0:1789762830.608019:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303844023568 time 303844105465 state 0x2 ref 0 00000100:00100000:3.0:1789762830.610285:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762830.610429:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067388672 00000100:00100000:0.0:1789762830.610448:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067388672 00000100:00100000:0.0:1789762830.610504:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303844023568 time 303844105465 state 0x0 ref 1 00000100:00100000:0.0:1789762830.610521:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303894105465 time 303844105465 state 0x1 ref 1 00000100:00100000:1.0:1789762830.647167:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303894105465 time 303894228208 state 0x0 ref 1 00000100:00100000:1.0:1789762830.647193:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067388672:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762830.647253:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 77594624, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2500b 00010000:00010000:1.0:1789762830.647262:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a9667c4/0xdf5a7ac59f175b40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.647293:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:1.0:1789762830.647297:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762830.647301:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a9667c4/0xdf5a7ac59f175b40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.647312:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a9667c4/0xdf5a7ac59f175b40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762830.659056:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967766 is committed 00002000:00010000:0.0:1789762830.659110:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:0.0:1789762830.659115:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 151563 -> 153611 00010000:00010000:0.0:1789762830.659125:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a9667c4/0xdf5a7ac59f175b40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.659135:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a9667c4/0xdf5a7ac59f175b40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.659142:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a9667c4/0xdf5a7ac59f175b40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.659147:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762830.659150:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a9667c4/0xdf5a7ac59f175b40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.659157:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a9667c4/0xdf5a7ac59f175b40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.659171:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a9667c4/0xdf5a7ac59f175b40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.659176:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a9667c4/0xdf5a7ac59f175b40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762830.659248:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067388672:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12062us (48963us total) trans 4294967766 rc 0/0 00000100:00100000:0.0:1789762830.659260:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 336 00000100:00100000:0.0:1789762830.659273:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303894105465 time 303894228208 state 0x2 ref 0 00000100:00100000:3.0:1789762830.661538:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762830.661594:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067388800 00000100:00100000:2.0:1789762830.661603:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067388800 00000100:00100000:2.0:1789762830.661659:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303894105465 time 303894228208 state 0x0 ref 1 00000100:00100000:2.0:1789762830.661674:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303944228208 time 303894228208 state 0x1 ref 1 00000100:00100000:0.0:1789762830.697267:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303944228208 time 303944336367 state 0x0 ref 1 00000100:00100000:0.0:1789762830.697287:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067388800:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762830.697329:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 78643200, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2580b 00010000:00010000:0.0:1789762830.697336:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175b47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.697348:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:0.0:1789762830.697352:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762830.697356:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175b47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.697367:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175b47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762830.709069:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 78643200 -> 79691776 00000001:00080000:2.0:1789762830.709074:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967767 is committed 00002000:00010000:0.0:1789762830.709075:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 153611 -> 155659 00010000:00010000:0.0:1789762830.709088:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175b47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.709109:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175b47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.709115:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175b47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.709120:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762830.709123:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175b47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.709131:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175b47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.709145:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175b47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.709150:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008bd9074b/0xdf5a7ac59f175b47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762830.709204:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067388800:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11923us (47667us total) trans 4294967767 rc 0/0 00000100:00100000:0.0:1789762830.709212:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 337 00000100:00100000:0.0:1789762830.709223:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303944228208 time 303944336367 state 0x2 ref 0 00000100:00100000:3.0:1789762830.711249:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762830.711283:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067388928 00000100:00100000:0.0:1789762830.711296:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067388928 00000100:00100000:0.0:1789762830.711330:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303944228208 time 303944336367 state 0x0 ref 1 00000100:00100000:0.0:1789762830.711340:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303994336367 time 303944336367 state 0x1 ref 1 00000100:00100000:1.0:1789762830.747350:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303994336367 time 303994416520 state 0x0 ref 1 00000100:00100000:1.0:1789762830.747373:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067388928:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762830.747422:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 79691776, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2600b 00010000:00010000:1.0:1789762830.747431:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175b4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.747446:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=303s v=(0 0) 00010000:00010000:1.0:1789762830.747450:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762830.747457:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175b4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.747471:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175b4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762830.759483:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967768 is committed 00002000:00010000:0.0:1789762830.759536:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 79691776 -> 80740352 00002000:00010000:0.0:1789762830.759541:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 155659 -> 157707 00010000:00010000:0.0:1789762830.759550:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175b4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.759559:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175b4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.759566:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175b4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.759573:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762830.759578:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175b4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.759588:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175b4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.759609:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175b4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.759630:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6e72ddb/0xdf5a7ac59f175b4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762830.759704:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067388928:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12337us (48453us total) trans 4294967768 rc 0/0 00000100:00100000:0.0:1789762830.759712:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 338 00000100:00100000:0.0:1789762830.759721:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303994336367 time 303994416520 state 0x2 ref 0 00000100:00100000:2.0:1789762830.761889:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762830.761945:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067389056 00000100:00100000:0.0:1789762830.761954:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067389056 00000100:00100000:0.0:1789762830.761990:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 303994336367 time 303994416520 state 0x0 ref 1 00000100:00100000:0.0:1789762830.762000:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304044416520 time 303994416520 state 0x1 ref 1 00000100:00100000:1.0:1789762830.797429:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304044416520 time 304044496187 state 0x0 ref 1 00000100:00100000:1.0:1789762830.797453:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067389056:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762830.797497:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 80740352, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2680b 00010000:00010000:1.0:1789762830.797506:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000452f7f5a/0xdf5a7ac59f175b55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.797521:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:1.0:1789762830.797525:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762830.797532:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000452f7f5a/0xdf5a7ac59f175b55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.797547:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000452f7f5a/0xdf5a7ac59f175b55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762830.809025:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967769 is committed 00002000:00010000:0.0:1789762830.809034:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 80740352 -> 81788928 00002000:00010000:0.0:1789762830.809039:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 157707 -> 159755 00010000:00010000:0.0:1789762830.809051:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000452f7f5a/0xdf5a7ac59f175b55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.809061:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000452f7f5a/0xdf5a7ac59f175b55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.809068:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000452f7f5a/0xdf5a7ac59f175b55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.809073:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762830.809077:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000452f7f5a/0xdf5a7ac59f175b55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.809095:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000452f7f5a/0xdf5a7ac59f175b55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.809110:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000452f7f5a/0xdf5a7ac59f175b55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.809115:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000452f7f5a/0xdf5a7ac59f175b55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762830.809180:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067389056:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11733us (47291us total) trans 4294967769 rc 0/0 00000100:00100000:0.0:1789762830.809188:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 339 00000100:00100000:0.0:1789762830.809197:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304044416520 time 304044496187 state 0x2 ref 0 00000100:00100000:3.0:1789762830.811343:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762830.811378:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067389184 00000100:00100000:0.0:1789762830.811386:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067389184 00000100:00100000:0.0:1789762830.811421:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304044416520 time 304044496187 state 0x0 ref 1 00000100:00100000:0.0:1789762830.811431:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304094496187 time 304044496187 state 0x1 ref 1 00000100:00100000:1.0:1789762830.847533:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304094496187 time 304094597045 state 0x0 ref 1 00000100:00100000:1.0:1789762830.847560:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067389184:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762830.847610:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 81788928, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2700b 00010000:00010000:1.0:1789762830.847632:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092a614ec/0xdf5a7ac59f175b5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.847648:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:1.0:1789762830.847653:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762830.847672:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000092a614ec/0xdf5a7ac59f175b5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.847686:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092a614ec/0xdf5a7ac59f175b5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762830.848029:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789762830.848051:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ca5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074677248:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762830.848106:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762830.848147:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074677248 00000100:00100000:3.0:1789762830.848154:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074677248 00000100:00100000:3.0:1789762830.848201:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:3.0:1789762830.848207:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074677248:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789762830.848234:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:3.0:1789762830.848277:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074677248:12345-0@lo:41:osp-pre-1-0.0 Request processed in 71us (172us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762830.848283:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:2.0:1789762830.848335:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ca5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074677248:0@lo:41:osp-pre-1-0.0 00000001:00080000:1.0:1789762830.859315:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967770 is committed 00002000:00010000:0.0:1789762830.859355:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 81788928 -> 82837504 00002000:00010000:0.0:1789762830.859361:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 159755 -> 161803 00010000:00010000:0.0:1789762830.859371:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092a614ec/0xdf5a7ac59f175b5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.859391:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000092a614ec/0xdf5a7ac59f175b5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.859397:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000092a614ec/0xdf5a7ac59f175b5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.859403:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762830.859406:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000092a614ec/0xdf5a7ac59f175b5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.859413:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000092a614ec/0xdf5a7ac59f175b5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.859428:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000092a614ec/0xdf5a7ac59f175b5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.859432:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092a614ec/0xdf5a7ac59f175b5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762830.859490:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067389184:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11938us (48146us total) trans 4294967770 rc 0/0 00000100:00100000:0.0:1789762830.859497:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 340 00000100:00100000:0.0:1789762830.859506:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304094496187 time 304094597045 state 0x2 ref 0 00000100:00100000:3.0:1789762830.861697:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762830.861741:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067389312 00000100:00100000:0.0:1789762830.861752:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067389312 00000100:00100000:0.0:1789762830.861784:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304094496187 time 304094597045 state 0x0 ref 1 00000100:00100000:0.0:1789762830.861794:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304144597045 time 304094597045 state 0x1 ref 1 00000100:00100000:2.0:1789762830.897627:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304144597045 time 304144678582 state 0x0 ref 1 00000100:00100000:2.0:1789762830.897655:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067389312:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762830.897703:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 82837504, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2780b 00010000:00010000:2.0:1789762830.897721:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175b63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.897736:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:2.0:1789762830.897740:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762830.897746:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175b63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.897760:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175b63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762830.909478:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 82837504 -> 83886080 00000001:00080000:3.0:1789762830.909481:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967771 is committed 00002000:00010000:2.0:1789762830.909486:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 161803 -> 163851 00010000:00010000:2.0:1789762830.909500:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175b63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.909513:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175b63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.909522:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175b63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.909530:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762830.909535:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175b63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.909545:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175b63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.909572:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175b63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.909580:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f175b63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762830.909661:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067389312:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12012us (47962us total) trans 4294967771 rc 0/0 00000100:00100000:2.0:1789762830.909671:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 341 00000100:00100000:2.0:1789762830.909682:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304144597045 time 304144678582 state 0x2 ref 0 00000100:00100000:3.0:1789762830.911736:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762830.911778:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067389440 00000100:00100000:0.0:1789762830.911786:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067389440 00000100:00100000:0.0:1789762830.911821:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304144597045 time 304144678582 state 0x0 ref 1 00000100:00100000:0.0:1789762830.911832:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304194678582 time 304144678582 state 0x1 ref 1 00000100:00100000:1.0:1789762830.947705:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304194678582 time 304194766655 state 0x0 ref 1 00000100:00100000:1.0:1789762830.947746:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067389440:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762830.947799:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 83886080, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2800b 00010000:00010000:1.0:1789762830.947809:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175b6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.947823:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:1.0:1789762830.947829:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762830.947836:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175b6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.947862:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175b6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762830.960663:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967772 is committed 00002000:00010000:0.0:1789762830.960677:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 83886080 -> 84934656 00002000:00010000:0.0:1789762830.960682:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 163851 -> 165899 00010000:00010000:0.0:1789762830.960693:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175b6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.960702:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175b6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.960708:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175b6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.960713:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762830.960717:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175b6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.960726:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175b6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.960756:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175b6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.960765:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175b6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762830.960831:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067389440:12345-192.168.203.37@tcp:4:dd.0 Request processed in 13095us (49094us total) trans 4294967772 rc 0/0 00000100:00100000:0.0:1789762830.960838:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 342 00000100:00100000:0.0:1789762830.960848:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304194678582 time 304194766655 state 0x2 ref 0 00000100:00100000:0.0:1789762830.963188:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762830.963243:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067389568 00000100:00100000:1.0:1789762830.963253:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067389568 00000100:00100000:1.0:1789762830.963300:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304194678582 time 304194766655 state 0x0 ref 1 00000100:00100000:1.0:1789762830.963316:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304244766655 time 304194766655 state 0x1 ref 1 00010000:00010000:1.0:1789762830.976051:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762830.997768:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304244766655 time 304244834372 state 0x0 ref 1 00000100:00100000:1.0:1789762830.997794:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067389568:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762830.997842:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 84934656, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2880b 00010000:00010000:1.0:1789762830.997851:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175b71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.997866:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:1.0:1789762830.997871:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762830.997878:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175b71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.997891:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175b71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.010145:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967773 is committed 00002000:00010000:0.0:1789762831.010192:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 84934656 -> 85983232 00002000:00010000:0.0:1789762831.010197:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 165899 -> 167947 00010000:00010000:0.0:1789762831.010207:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175b71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.010216:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175b71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.010223:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175b71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.010228:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.010231:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175b71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.010246:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175b71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.010259:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175b71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.010265:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175b71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.010324:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067389568:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12539us (47135us total) trans 4294967773 rc 0/0 00000100:00100000:0.0:1789762831.010332:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 343 00000100:00100000:0.0:1789762831.010341:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304244766655 time 304244834372 state 0x2 ref 0 00000100:00100000:3.0:1789762831.012848:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762831.012905:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067389696 00000100:00100000:2.0:1789762831.012913:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067389696 00000100:00100000:2.0:1789762831.012987:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304244766655 time 304244834372 state 0x0 ref 1 00000100:00080000:1.0:1789762831.012993:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762831.013003:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304294834372 time 304244834372 state 0x1 ref 1 00000100:00100000:3.0:1789762831.047840:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304294834372 time 304294906153 state 0x0 ref 1 00000100:00100000:3.0:1789762831.047881:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067389696:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762831.047963:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 85983232, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2900b 00010000:00010000:3.0:1789762831.047971:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4bf1399/0xdf5a7ac59f175b78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.047985:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:3.0:1789762831.047989:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762831.047994:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4bf1399/0xdf5a7ac59f175b78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.048007:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4bf1399/0xdf5a7ac59f175b78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762831.064302:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967774 is committed 00002000:00010000:0.0:1789762831.064329:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 85983232 -> 87031808 00002000:00010000:0.0:1789762831.064336:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 167947 -> 169995 00010000:00010000:0.0:1789762831.064349:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4bf1399/0xdf5a7ac59f175b78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.064358:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f4bf1399/0xdf5a7ac59f175b78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.064365:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f4bf1399/0xdf5a7ac59f175b78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.064380:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d92e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.064383:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4bf1399/0xdf5a7ac59f175b78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.064391:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4bf1399/0xdf5a7ac59f175b78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.064405:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f4bf1399/0xdf5a7ac59f175b78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.064410:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4bf1399/0xdf5a7ac59f175b78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.064476:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067389696:12345-192.168.203.37@tcp:4:dd.0 Request processed in 16602us (51626us total) trans 4294967774 rc 0/0 00000100:00100000:0.0:1789762831.064484:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 344 00000100:00100000:0.0:1789762831.064494:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304294834372 time 304294906153 state 0x2 ref 0 00000100:00100000:3.0:1789762831.066851:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.066886:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067389824 00000100:00100000:0.0:1789762831.066893:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067389824 00000100:00100000:0.0:1789762831.066953:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304294834372 time 304294906153 state 0x0 ref 1 00000100:00100000:0.0:1789762831.066969:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304344906153 time 304294906153 state 0x1 ref 1 00000100:00100000:1.0:1789762831.097965:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304344906153 time 304345026484 state 0x0 ref 1 00000100:00100000:1.0:1789762831.097992:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067389824:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762831.098065:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 87031808, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2980b 00010000:00010000:1.0:1789762831.098075:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175b7f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.098109:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:1.0:1789762831.098115:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.098123:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175b7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.098137:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175b7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762831.109985:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 87031808 -> 88080384 00000001:00080000:1.0:1789762831.109985:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967775 is committed 00002000:00010000:0.0:1789762831.109996:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 169995 -> 172043 00010000:00010000:0.0:1789762831.110008:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175b7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.110018:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175b7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.110025:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175b7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.110030:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.110033:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175b7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.110040:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175b7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.110055:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175b7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.110059:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175b7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.110120:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067389824:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12137us (43269us total) trans 4294967775 rc 0/0 00000100:00100000:0.0:1789762831.110129:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 345 00000100:00100000:0.0:1789762831.110139:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304344906153 time 304345026484 state 0x2 ref 0 00000100:00100000:3.0:1789762831.112359:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.112394:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067389952 00000100:00100000:0.0:1789762831.112402:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067389952 00000100:00100000:0.0:1789762831.112438:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304344906153 time 304345026484 state 0x0 ref 1 00000100:00100000:0.0:1789762831.112449:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304395026484 time 304345026484 state 0x1 ref 1 00000100:00100000:0.0:1789762831.148028:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304395026484 time 304395092990 state 0x0 ref 1 00000100:00100000:0.0:1789762831.148053:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067389952:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762831.148102:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 88080384, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2a00b 00010000:00010000:0.0:1789762831.148109:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175b86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.148122:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:0.0:1789762831.148126:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.148130:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175b86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.148141:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175b86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.160494:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967776 is committed 00002000:00010000:0.0:1789762831.160551:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 88080384 -> 89128960 00002000:00010000:0.0:1789762831.160555:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 172043 -> 174091 00010000:00010000:0.0:1789762831.160565:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175b86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.160584:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175b86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.160591:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175b86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.160596:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.160599:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175b86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.160607:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175b86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.160633:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175b86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.160639:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acb3d8cf/0xdf5a7ac59f175b86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.160697:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067389952:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12652us (48337us total) trans 4294967776 rc 0/0 00000100:00100000:0.0:1789762831.160706:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 346 00000100:00100000:0.0:1789762831.160715:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304395026484 time 304395092990 state 0x2 ref 0 00000100:00100000:3.0:1789762831.163020:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.163055:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067390080 00000100:00100000:0.0:1789762831.163068:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067390080 00000100:00100000:0.0:1789762831.163106:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304395026484 time 304395092990 state 0x0 ref 1 00000100:00100000:0.0:1789762831.163117:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304445092990 time 304395092990 state 0x1 ref 1 00000100:00100000:1.0:1789762831.198139:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304445092990 time 304445199562 state 0x0 ref 1 00000100:00100000:1.0:1789762831.198172:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067390080:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762831.198247:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 89128960, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x2a80b 00010000:00010000:1.0:1789762831.198257:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.198273:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:1.0:1789762831.198277:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.198285:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.198300:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762831.210142:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 89128960 -> 90177536 00000001:00080000:0.0:1789762831.210149:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967777 is committed 00002000:00010000:1.0:1789762831.210157:0:14171:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb mtime from disk: 1789762830 -> 1789762831 00002000:00010000:1.0:1789762831.210161:0:14171:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb ctime from disk: 1789762830 -> 1789762831 00002000:00010000:1.0:1789762831.210164:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 174091 -> 176139 00010000:00010000:1.0:1789762831.210178:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.210192:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.210201:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.210210:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762831.210214:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.210224:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.210244:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.210252:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762831.210332:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067390080:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12170us (47309us total) trans 4294967777 rc 0/0 00000100:00100000:1.0:1789762831.210353:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 347 00000100:00100000:1.0:1789762831.210366:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304445092990 time 304445199562 state 0x2 ref 0 00000100:00100000:3.0:1789762831.212699:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.212749:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067390208 00000100:00100000:0.0:1789762831.212757:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067390208 00000100:00100000:0.0:1789762831.212797:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304445092990 time 304445199562 state 0x0 ref 1 00000100:00100000:0.0:1789762831.212809:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304495199562 time 304445199562 state 0x1 ref 1 00000100:00100000:2.0:1789762831.248211:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304495199562 time 304495276240 state 0x0 ref 1 00000100:00100000:2.0:1789762831.248236:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067390208:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762831.248284:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 90177536, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x2b00b 00010000:00010000:2.0:1789762831.248291:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.248319:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:2.0:1789762831.248322:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762831.248327:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.248337:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762831.259644:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967778 is committed 00002000:00010000:0.0:1789762831.259656:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 90177536 -> 91226112 00002000:00010000:0.0:1789762831.259661:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 176139 -> 178187 00010000:00010000:0.0:1789762831.259673:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.259684:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.259690:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.259695:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.259698:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.259715:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.259730:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.259735:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f175b94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.259804:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067390208:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11575us (47102us total) trans 4294967778 rc 0/0 00000100:00100000:0.0:1789762831.259812:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 348 00000100:00100000:0.0:1789762831.259822:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304495199562 time 304495276240 state 0x2 ref 0 00000100:00100000:0.0:1789762831.262408:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762831.262451:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067390336 00000100:00100000:1.0:1789762831.262459:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067390336 00000100:00100000:1.0:1789762831.262500:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304495199562 time 304495276240 state 0x0 ref 1 00000100:00100000:1.0:1789762831.262511:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304545276240 time 304495276240 state 0x1 ref 1 00000100:00100000:1.0:1789762831.298296:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304545276240 time 304545357174 state 0x0 ref 1 00000100:00100000:1.0:1789762831.298324:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067390336:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762831.298377:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 91226112, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x2b80b 00010000:00010000:1.0:1789762831.298388:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.298405:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:1.0:1789762831.298410:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.298417:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.298432:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.309292:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967779 is committed 00002000:00010000:0.0:1789762831.309326:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 91226112 -> 92274688 00002000:00010000:0.0:1789762831.309331:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 178187 -> 180235 00010000:00010000:0.0:1789762831.309341:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.309350:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.309356:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.309372:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.309376:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.309382:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.309395:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.309401:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175b9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.309463:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067390336:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11146us (47053us total) trans 4294967779 rc 0/0 00000100:00100000:0.0:1789762831.309471:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 349 00000100:00100000:0.0:1789762831.309481:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304545276240 time 304545357174 state 0x2 ref 0 00000100:00100000:2.0:1789762831.311682:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762831.311738:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067390464 00000100:00100000:1.0:1789762831.311748:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067390464 00000100:00100000:1.0:1789762831.311794:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304545276240 time 304545357174 state 0x0 ref 1 00000100:00100000:1.0:1789762831.311809:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304595357174 time 304545357174 state 0x1 ref 1 00000100:00100000:1.0:1789762831.348373:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304595357174 time 304595436757 state 0x0 ref 1 00000100:00100000:1.0:1789762831.348401:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067390464:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762831.348454:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 92274688, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x2c00b 00010000:00010000:1.0:1789762831.348463:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175ba2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.348479:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:1.0:1789762831.348484:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.348491:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175ba2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.348506:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175ba2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.361315:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967780 is committed 00002000:00010000:0.0:1789762831.361343:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 92274688 -> 93323264 00002000:00010000:0.0:1789762831.361348:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 180235 -> 182283 00010000:00010000:0.0:1789762831.361360:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175ba2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.361370:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175ba2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.361388:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175ba2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.361393:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.361398:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175ba2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.361404:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175ba2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.361419:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175ba2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.361425:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175ba2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.361499:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067390464:12345-192.168.203.37@tcp:4:dd.0 Request processed in 13105us (49815us total) trans 4294967780 rc 0/0 00000100:00100000:0.0:1789762831.361508:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 350 00000100:00100000:0.0:1789762831.361518:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304595357174 time 304595436757 state 0x2 ref 0 00000100:00100000:2.0:1789762831.364014:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.364048:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067390592 00000100:00100000:0.0:1789762831.364056:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067390592 00000100:00100000:0.0:1789762831.364097:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304595357174 time 304595436757 state 0x0 ref 1 00000100:00100000:0.0:1789762831.364118:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304645436757 time 304595436757 state 0x1 ref 1 00000100:00100000:0.0:1789762831.398422:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304645436757 time 304645490908 state 0x0 ref 1 00000100:00100000:0.0:1789762831.398442:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067390592:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762831.398483:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 93323264, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x2c80b 00010000:00010000:0.0:1789762831.398491:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175ba9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.398504:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:0.0:1789762831.398507:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.398512:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175ba9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.398522:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175ba9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.409575:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967781 is committed 00002000:00010000:0.0:1789762831.409647:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 93323264 -> 94371840 00002000:00010000:0.0:1789762831.409653:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 182283 -> 184331 00010000:00010000:0.0:1789762831.409665:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175ba9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.409676:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175ba9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.409700:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175ba9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.409708:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.409713:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175ba9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.409722:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175ba9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.409739:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175ba9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.409747:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175ba9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.409816:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067390592:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11377us (45799us total) trans 4294967781 rc 0/0 00000100:00100000:0.0:1789762831.409826:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 351 00000100:00100000:0.0:1789762831.409837:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304645436757 time 304645490908 state 0x2 ref 0 00000100:00100000:2.0:1789762831.411948:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.411973:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067390720 00000100:00100000:0.0:1789762831.411981:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067390720 00000100:00100000:0.0:1789762831.412018:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304645436757 time 304645490908 state 0x0 ref 1 00000100:00100000:0.0:1789762831.412037:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304695490908 time 304645490908 state 0x1 ref 1 00000100:00100000:0.0:1789762831.448475:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304695490908 time 304695542280 state 0x0 ref 1 00000100:00100000:0.0:1789762831.448497:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067390720:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762831.448552:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 94371840, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x2d00b 00010000:00010000:0.0:1789762831.448560:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175bb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.448572:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:0.0:1789762831.448576:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.448580:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175bb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.448592:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175bb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.464489:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967782 is committed 00002000:00010000:0.0:1789762831.464515:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 94371840 -> 95420416 00002000:00010000:0.0:1789762831.464520:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 184331 -> 186379 00010000:00010000:0.0:1789762831.464532:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175bb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.464541:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175bb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.464559:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175bb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.464564:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.464567:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175bb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.464573:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175bb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.464588:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175bb0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.464593:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038b13339/0xdf5a7ac59f175bb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.464670:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067390720:12345-192.168.203.37@tcp:4:dd.0 Request processed in 16178us (52722us total) trans 4294967782 rc 0/0 00000100:00100000:0.0:1789762831.464677:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 352 00000100:00100000:0.0:1789762831.464686:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304695490908 time 304695542280 state 0x2 ref 0 00000100:00100000:3.0:1789762831.466975:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.467008:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067390848 00000100:00100000:0.0:1789762831.467016:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067390848 00000100:00100000:0.0:1789762831.467052:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304695490908 time 304695542280 state 0x0 ref 1 00000100:00100000:0.0:1789762831.467072:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304745542280 time 304695542280 state 0x1 ref 1 00000100:00100000:1.0:1789762831.498567:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304745542280 time 304745633059 state 0x0 ref 1 00000100:00100000:1.0:1789762831.498591:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067390848:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762831.498665:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 95420416, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x2d80b 00010000:00010000:1.0:1789762831.498674:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bb7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.498687:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:1.0:1789762831.498690:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.498695:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.498706:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.510182:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967783 is committed 00002000:00010000:0.0:1789762831.510202:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 95420416 -> 96468992 00002000:00010000:0.0:1789762831.510208:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 186379 -> 188427 00010000:00010000:0.0:1789762831.510217:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.510226:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bb7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.510234:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.510239:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.510243:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.510250:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bb7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.510263:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bb7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.510268:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bb7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.510335:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067390848:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11751us (43358us total) trans 4294967783 rc 0/0 00000100:00100000:0.0:1789762831.510346:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 353 00000100:00100000:0.0:1789762831.510358:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304745542280 time 304745633059 state 0x2 ref 0 00000100:00100000:3.0:1789762831.512458:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.512491:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067390976 00000100:00100000:0.0:1789762831.512499:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067390976 00000100:00100000:0.0:1789762831.512535:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304745542280 time 304745633059 state 0x0 ref 1 00000100:00100000:0.0:1789762831.512545:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304795633059 time 304745633059 state 0x1 ref 1 00000100:00100000:1.0:1789762831.548689:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304795633059 time 304795757007 state 0x0 ref 1 00000100:00100000:1.0:1789762831.548712:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067390976:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762831.548757:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 96468992, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x2e00b 00010000:00010000:1.0:1789762831.548763:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bbe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.548777:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:1.0:1789762831.548781:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.548785:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.548796:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.560516:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967784 is committed 00002000:00010000:0.0:1789762831.560533:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 96468992 -> 97517568 00002000:00010000:0.0:1789762831.560538:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 188427 -> 190475 00010000:00010000:0.0:1789762831.560549:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.560558:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bbe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.560565:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.560570:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.560574:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.560591:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bbe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.560605:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bbe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.560622:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee13fa9/0xdf5a7ac59f175bbe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.560677:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067390976:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11973us (48219us total) trans 4294967784 rc 0/0 00000100:00100000:0.0:1789762831.560686:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 354 00000100:00100000:0.0:1789762831.560695:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304795633059 time 304795757007 state 0x2 ref 0 00000100:00100000:2.0:1789762831.562851:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.562889:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067391104 00000100:00100000:0.0:1789762831.562896:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067391104 00000100:00100000:0.0:1789762831.562949:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304795633059 time 304795757007 state 0x0 ref 1 00000100:00100000:0.0:1789762831.562960:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304845757007 time 304795757007 state 0x1 ref 1 00000100:00100000:2.0:1789762831.598763:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304845757007 time 304845828641 state 0x0 ref 1 00000100:00100000:2.0:1789762831.598786:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067391104:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762831.598831:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 97517568, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x2e80b 00010000:00010000:2.0:1789762831.598840:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175bc5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.598856:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:2.0:1789762831.598860:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762831.598867:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175bc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.598881:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175bc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.609570:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967785 is committed 00002000:00010000:0.0:1789762831.609596:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 97517568 -> 98566144 00002000:00010000:0.0:1789762831.609602:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 190475 -> 192523 00010000:00010000:0.0:1789762831.609628:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175bc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.609655:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175bc5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.609665:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175bc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.609674:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.609679:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175bc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.609690:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175bc5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.609707:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175bc5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.609715:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d885e6/0xdf5a7ac59f175bc5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.609769:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067391104:12345-192.168.203.37@tcp:4:dd.0 Request processed in 10991us (46920us total) trans 4294967785 rc 0/0 00000100:00100000:0.0:1789762831.609777:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 355 00000100:00100000:0.0:1789762831.609785:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304845757007 time 304845828641 state 0x2 ref 0 00000100:00100000:0.0:1789762831.611963:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762831.612016:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067391232 00000100:00100000:1.0:1789762831.612024:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067391232 00000100:00100000:1.0:1789762831.612068:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304845757007 time 304845828641 state 0x0 ref 1 00000100:00100000:1.0:1789762831.612084:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304895828641 time 304845828641 state 0x1 ref 1 00000100:00100000:1.0:1789762831.648815:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304895828641 time 304895881154 state 0x0 ref 1 00000100:00100000:1.0:1789762831.648838:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067391232:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762831.648886:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 98566144, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x2f00b 00010000:00010000:1.0:1789762831.648892:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175bcc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.648905:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:1.0:1789762831.648909:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.648938:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175bcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.648956:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175bcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.660182:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967786 is committed 00002000:00010000:0.0:1789762831.660240:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 98566144 -> 99614720 00002000:00010000:0.0:1789762831.660246:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 192523 -> 194571 00010000:00010000:0.0:1789762831.660258:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175bcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.660270:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175bcc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.660281:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175bcc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.660289:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.660294:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175bcc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.660304:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175bcc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.660322:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175bcc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.660331:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086169fb5/0xdf5a7ac59f175bcc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.660395:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067391232:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11563us (48430us total) trans 4294967786 rc 0/0 00000100:00100000:0.0:1789762831.660407:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 356 00000100:00100000:0.0:1789762831.660426:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304895828641 time 304895881154 state 0x2 ref 0 00000100:00100000:2.0:1789762831.662576:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.662623:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067391360 00000100:00100000:0.0:1789762831.662631:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067391360 00000100:00100000:0.0:1789762831.662666:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304895828641 time 304895881154 state 0x0 ref 1 00000100:00100000:0.0:1789762831.662677:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304945881154 time 304895881154 state 0x1 ref 1 00000100:00100000:1.0:1789762831.698908:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304945881154 time 304945970379 state 0x0 ref 1 00000100:00100000:1.0:1789762831.698959:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067391360:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762831.699026:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 99614720, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x2f80b 00010000:00010000:1.0:1789762831.699035:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175bd3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.699050:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:1.0:1789762831.699055:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.699062:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175bd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.699078:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175bd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.711250:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967787 is committed 00002000:00010000:0.0:1789762831.711274:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 99614720 -> 100663296 00002000:00010000:0.0:1789762831.711280:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 194571 -> 196619 00010000:00010000:0.0:1789762831.711292:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175bd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.711306:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175bd3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.711317:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175bd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.711325:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.711330:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175bd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.711341:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175bd3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.711359:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175bd3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.711376:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000424c73a0/0xdf5a7ac59f175bd3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.711437:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067391360:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12488us (48861us total) trans 4294967787 rc 0/0 00000100:00100000:0.0:1789762831.711445:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 357 00000100:00100000:0.0:1789762831.711454:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304945881154 time 304945970379 state 0x2 ref 0 00000100:00100000:0.0:1789762831.713720:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762831.713773:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067391488 00000100:00100000:1.0:1789762831.713782:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067391488 00000100:00100000:1.0:1789762831.713824:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304945881154 time 304945970379 state 0x0 ref 1 00000100:00100000:1.0:1789762831.713840:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304995970379 time 304945970379 state 0x1 ref 1 00000100:00100000:0.0:1789762831.749011:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304995970379 time 304996079965 state 0x0 ref 1 00000100:00100000:0.0:1789762831.749031:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067391488:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762831.749072:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 100663296, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x3000b 00010000:00010000:0.0:1789762831.749079:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175bda lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.749092:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=304s v=(0 0) 00010000:00010000:0.0:1789762831.749095:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.749100:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175bda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.749111:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175bda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.760461:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967788 is committed 00002000:00010000:0.0:1789762831.760477:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 100663296 -> 101711872 00002000:00010000:0.0:1789762831.760482:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 196619 -> 198667 00010000:00010000:0.0:1789762831.760495:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175bda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.760509:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175bda lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.760520:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175bda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.760528:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.760533:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175bda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.760543:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175bda lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.760562:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175bda lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.760571:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042618884/0xdf5a7ac59f175bda lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.760645:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067391488:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11617us (46923us total) trans 4294967788 rc 0/0 00000100:00100000:0.0:1789762831.760661:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 358 00000100:00100000:0.0:1789762831.760674:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304995970379 time 304996079965 state 0x2 ref 0 00000100:00100000:2.0:1789762831.762943:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.762969:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067391616 00000100:00100000:0.0:1789762831.762979:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067391616 00000100:00100000:0.0:1789762831.763021:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 304995970379 time 304996079965 state 0x0 ref 1 00000100:00100000:0.0:1789762831.763035:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305046079965 time 304996079965 state 0x1 ref 1 00000100:00100000:1.0:1789762831.799104:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305046079965 time 305046172684 state 0x0 ref 1 00000100:00100000:1.0:1789762831.799125:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067391616:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762831.799166:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 101711872, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x3080b 00010000:00010000:1.0:1789762831.799173:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175be1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.799201:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=305s v=(0 0) 00010000:00010000:1.0:1789762831.799205:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.799212:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175be1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.799228:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175be1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.811192:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967789 is committed 00002000:00010000:0.0:1789762831.811256:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 101711872 -> 102760448 00002000:00010000:0.0:1789762831.811262:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 198667 -> 200715 00010000:00010000:0.0:1789762831.811274:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175be1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.811287:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175be1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.811298:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175be1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.811307:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.811312:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175be1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.811320:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175be1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.811341:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175be1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.811347:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000217e9d3b/0xdf5a7ac59f175be1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.811407:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067391616:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12287us (48488us total) trans 4294967789 rc 0/0 00000100:00100000:0.0:1789762831.811414:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 359 00000100:00100000:0.0:1789762831.811423:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305046079965 time 305046172684 state 0x2 ref 0 00000100:00100000:2.0:1789762831.813708:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.813755:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067391744 00000100:00100000:0.0:1789762831.813763:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067391744 00000100:00100000:0.0:1789762831.813800:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305046079965 time 305046172684 state 0x0 ref 1 00000100:00100000:0.0:1789762831.813812:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305096172684 time 305046172684 state 0x1 ref 1 00000100:00100000:0.0:1789762831.849161:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305096172684 time 305096227975 state 0x0 ref 1 00000100:00100000:0.0:1789762831.849182:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067391744:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762831.849226:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 102760448, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x3100b 00010000:00010000:0.0:1789762831.849233:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175be8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.849261:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=305s v=(0 0) 00010000:00010000:0.0:1789762831.849265:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.849269:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175be8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.849280:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175be8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.861108:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967790 is committed 00002000:00010000:0.0:1789762831.861135:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 102760448 -> 103809024 00002000:00010000:0.0:1789762831.861140:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 200715 -> 202763 00010000:00010000:0.0:1789762831.861153:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175be8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.861167:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175be8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.861177:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175be8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.861185:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.861189:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175be8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.861196:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175be8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.861223:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175be8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.861229:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039096ca9/0xdf5a7ac59f175be8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.861291:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067391744:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12114us (47583us total) trans 4294967790 rc 0/0 00000100:00100000:0.0:1789762831.861298:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 360 00000100:00100000:0.0:1789762831.861307:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305096172684 time 305096227975 state 0x2 ref 0 00000100:00100000:2.0:1789762831.863757:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.863797:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067391872 00000100:00100000:0.0:1789762831.863805:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067391872 00000100:00100000:0.0:1789762831.863843:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305096172684 time 305096227975 state 0x0 ref 1 00000100:00100000:0.0:1789762831.863853:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305146227975 time 305096227975 state 0x1 ref 1 00000100:00100000:3.0:1789762831.872073:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789762831.872089:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074677376:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762831.872128:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762831.872177:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074677376 00000100:00100000:2.0:1789762831.872183:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074677376 00000100:00100000:2.0:1789762831.872227:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 281 00000100:00100000:2.0:1789762831.872238:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074677376:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762831.872272:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762831.872320:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074677376:12345-0@lo:41:osp-pre-0-1.0 Request processed in 83us (188us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762831.872329:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 281 00000100:00100000:3.0:1789762831.872354:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074677376:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762831.899252:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305146227975 time 305146319483 state 0x0 ref 1 00000100:00100000:1.0:1789762831.899276:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067391872:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762831.899328:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 103809024, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x3180b 00010000:00010000:1.0:1789762831.899335:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175bef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.899349:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=305s v=(0 0) 00010000:00010000:1.0:1789762831.899352:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.899357:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175bef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.899369:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175bef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.910556:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967791 is committed 00002000:00010000:0.0:1789762831.910578:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb size from disk: 103809024 -> 104857600 00002000:00010000:0.0:1789762831.910583:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x6:0x0] updating lvb blocks from disk: 202763 -> 204811 00010000:00010000:0.0:1789762831.910593:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175bef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.910602:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175bef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.910632:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175bef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.910638:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9231400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.910643:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175bef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.910649:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175bef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.910664:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175bef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.910669:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f132762a/0xdf5a7ac59f175bef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.910729:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067391872:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11461us (46972us total) trans 4294967791 rc 0/0 00000100:00100000:0.0:1789762831.910738:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 361 00000100:00100000:0.0:1789762831.910748:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305146227975 time 305146319483 state 0x2 ref 0 00000100:00100000:0.0:1789762831.912240:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762831.912279:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067392128 00000100:00100000:3.0:1789762831.912287:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067392128 00000100:00100000:3.0:1789762831.912340:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 64 00000100:00080000:1.0:1789762831.912346:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762831.912348:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbdc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:13997:x1876702067392128:12345-192.168.203.37@tcp:35:dd.0 00000001:00080000:3.0:1789762831.912523:0:6421:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967393 00000100:00100000:3.0:1789762831.912587:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbdc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:13997:x1876702067392128:12345-192.168.203.37@tcp:35:dd.0 Request processed in 240us (348us total) trans 4294967394 rc 0/0 00000100:00100000:3.0:1789762831.912594:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 64 00000020:00080000:3.0:1789762831.912597:0:6421:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789762831.935052:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762831.935112:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067392256 00000100:00100000:3.0:1789762831.935121:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067392256 00000100:00100000:3.0:1789762831.935168:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 42 00000100:00100000:3.0:1789762831.935176:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c91450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:6358:x1876702067392256:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762831.935187:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1789762831.935202:0:6406:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x8:0x0] updating lvb blocks from disk: 1 -> 16391 00010000:00010000:3.0:1789762831.935221:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000026b0307a/0xdf5a7ac59f175796 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c76e expref: 6 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.935240:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762831.935277:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c91450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:6358:x1876702067392256:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 103us (232us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762831.935283:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 42 00000100:00100000:2.0:1789762831.955051:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762831.955096:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067392512 00000100:00100000:3.0:1789762831.955103:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067392512 00000100:00100000:3.0:1789762831.955146:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 241 00000100:00100000:3.0:1789762831.955154:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:14013:x1876702067392512:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762831.955173:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762831.955188:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93c00/0xdf5a7ac59f175bf6 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7c9 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.955284:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175bfd lrc: 3/1,0 mode: --/CR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.955296:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175bfd lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.955305:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175bfd lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762831.955313:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14:0x0]: returning size 104857600 00010000:00010000:3.0:1789762831.955404:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d93c00/0xdf5a7ac59f175bf6 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x12:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7c9 expref: 23 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.955416:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175bfd lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7c9 expref: 22 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.955422:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff9c37d3beef00) 00010000:00010000:3.0:1789762831.955425:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:3.0:1789762831.955427:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d92400, rc 0) 00000100:00100000:3.0:1789762831.955465:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:14013:x1876702067392512:12345-192.168.203.37@tcp:101:dd.0 Request processed in 313us (423us total) trans 4294967395 rc 0/0 00000100:00100000:3.0:1789762831.955470:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 241 00000100:00100000:0.0:1789762831.956974:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762831.957007:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067392640 00000100:00100000:3.0:1789762831.957012:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067392640 00000100:00100000:3.0:1789762831.957041:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 152 00000100:00100000:3.0:1789762831.957047:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067392640:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762831.957053:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762831.957069:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de12dad1/0xdf5a7ac59f175c0b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7d0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762831.957085:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 104857600, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x3200b 00010000:00010000:3.0:1789762831.957096:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=305s v=(0 0) 00010000:00010000:3.0:1789762831.957100:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762831.957105:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de12dad1/0xdf5a7ac59f175c0b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7d0 expref: 6 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.957115:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000de12dad1/0xdf5a7ac59f175c0b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7d0 expref: 6 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.957123:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37f7d92200, rc 0) 00000100:00100000:3.0:1789762831.957148:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067392640:12345-192.168.203.37@tcp:101:dd.0 Request processed in 102us (176us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762831.957153:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 152 00000100:00100000:2.0:1789762831.959363:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.959438:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067392768 00000100:00100000:0.0:1789762831.959444:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067392768 00000100:00100000:0.0:1789762831.959482:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 2 deadline 299921444915 time 299821464962 state 0x0 ref 1 00000100:00100000:0.0:1789762831.959493:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 2 deadline 299921464962 time 299821464962 state 0x1 ref 1 00000100:00100000:0.0:1789762831.959515:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 2 deadline 299921464962 time 305206593536 state 0x0 ref 1 00000100:00100000:0.0:1789762831.959525:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067392768:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762831.959542:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c12 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.959551:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.959554:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.959557:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c12 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.959565:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c12 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.963040:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c12 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.963051:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c12 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.963058:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c12 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.963075:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762831.963079:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c12 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.963085:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c12 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.963098:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c12 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.963103:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c12 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.963152:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067392768:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3626us (3787us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762831.963162:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 362 00000100:00100000:0.0:1789762831.963173:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 2 deadline 299921464962 time 305206593536 state 0x2 ref 0 00000100:00100000:0.0:1789762831.964899:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762831.964947:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067392896 00000100:00100000:1.0:1789762831.964953:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067392896 00000100:00100000:1.0:1789762831.964981:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 2 deadline 299921464962 time 305206593536 state 0x0 ref 1 00000100:00100000:1.0:1789762831.964990:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 2 deadline 305306593536 time 305206593536 state 0x1 ref 1 00000100:00100000:1.0:1789762831.965017:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 1 deadline 305306593536 time 305212095755 state 0x0 ref 1 00000100:00100000:1.0:1789762831.965026:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067392896:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762831.965047:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c19 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.965060:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.965064:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.965069:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c19 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.965081:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c19 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.967309:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c19 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.967318:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c19 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.967324:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c19 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.967335:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762831.967338:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c19 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.967344:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c19 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.967354:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c19 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.967359:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c19 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762831.967397:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067392896:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2371us (2498us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762831.967406:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 363 00000100:00100000:1.0:1789762831.967414:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 1 deadline 305306593536 time 305212095755 state 0x2 ref 0 00000100:00100000:2.0:1789762831.969500:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.969561:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067393024 00000100:00100000:0.0:1789762831.969569:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067393024 00000100:00100000:0.0:1789762831.969633:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 1 deadline 305306593536 time 305212095755 state 0x0 ref 1 00000100:00100000:0.0:1789762831.969648:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 1 deadline 305312095755 time 305212095755 state 0x1 ref 1 00000100:00100000:0.0:1789762831.969698:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305312095755 time 305216773564 state 0x0 ref 1 00000100:00100000:0.0:1789762831.969712:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067393024:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762831.969741:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c20 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.969758:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.969763:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.969770:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c20 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.969783:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c20 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.972419:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c20 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.972432:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c20 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.972439:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c20 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.972444:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762831.972448:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c20 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.972463:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c20 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.972479:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c20 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.972484:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000805fd054/0xdf5a7ac59f175c20 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762831.972529:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067393024:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2819us (3033us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762831.972537:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 364 00000100:00100000:3.0:1789762831.972549:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305312095755 time 305216773564 state 0x2 ref 0 00000100:00100000:2.0:1789762831.974675:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762831.974721:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067393152 00000100:00100000:0.0:1789762831.974728:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067393152 00000100:00100000:0.0:1789762831.974765:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305312095755 time 305216773564 state 0x0 ref 1 00000100:00100000:0.0:1789762831.974775:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305316773564 time 305216773564 state 0x1 ref 1 00010000:00010000:3.0:1789762832.000051:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762832.069762:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305316773564 time 305316828554 state 0x0 ref 1 00000100:00100000:0.0:1789762832.069788:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067393152:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762832.069825:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f175c27 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.069852:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762832.069855:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762832.069861:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f175c27 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.069872:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f175c27 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.072818:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f175c27 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.072831:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f175c27 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.072841:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f175c27 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.072856:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762832.072861:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f175c27 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.072872:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f175c27 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.072891:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f175c27 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.072899:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8c3f943/0xdf5a7ac59f175c27 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762832.072986:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067393152:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3206us (98311us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762832.072996:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 365 00000100:00100000:1.0:1789762832.073007:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305316773564 time 305316828554 state 0x2 ref 0 00000100:00100000:2.0:1789762832.075089:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762832.075148:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067393280 00000100:00100000:0.0:1789762832.075157:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067393280 00000100:00100000:0.0:1789762832.075193:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305316773564 time 305316828554 state 0x0 ref 1 00000100:00100000:0.0:1789762832.075205:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305416828554 time 305316828554 state 0x1 ref 1 00000100:00100000:1.0:1789762832.169860:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305416828554 time 305416921751 state 0x0 ref 1 00000100:00100000:1.0:1789762832.169890:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067393280:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762832.169956:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c2e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.169994:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762832.169999:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762832.170006:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c2e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.170021:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c2e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.172983:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c2e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.172998:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c2e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.173008:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c2e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.173016:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762832.173022:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c2e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.173031:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c2e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.173050:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c2e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.173071:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c2e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762832.173130:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067393280:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3249us (98038us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762832.173140:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 366 00000100:00100000:1.0:1789762832.173151:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305416828554 time 305416921751 state 0x2 ref 0 00000100:00100000:2.0:1789762832.175181:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762832.175230:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067393408 00000100:00100000:3.0:1789762832.175239:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067393408 00000100:00100000:3.0:1789762832.175285:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305416828554 time 305416921751 state 0x0 ref 1 00000100:00100000:3.0:1789762832.175301:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305516921751 time 305416921751 state 0x1 ref 1 00000100:00100000:1.0:1789762832.269974:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305516921751 time 305517038739 state 0x0 ref 1 00000100:00100000:1.0:1789762832.270000:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067393408:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762832.270036:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c35 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.270050:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762832.270054:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762832.270059:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c35 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.270070:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c35 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.272735:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c35 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.272757:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c35 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.272764:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c35 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.272769:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762832.272774:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c35 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.272781:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c35 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.272794:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c35 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.272799:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c35 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762832.272845:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067393408:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2856us (97665us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762832.272853:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 367 00000100:00100000:1.0:1789762832.272860:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305516921751 time 305517038739 state 0x2 ref 0 00000100:00100000:0.0:1789762832.274681:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762832.274718:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067393536 00000100:00100000:1.0:1789762832.274732:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067393536 00000100:00100000:1.0:1789762832.274765:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305516921751 time 305517038739 state 0x0 ref 1 00000100:00100000:1.0:1789762832.274775:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305617038739 time 305517038739 state 0x1 ref 1 00000100:00100000:1.0:1789762832.370041:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305617038739 time 305617106574 state 0x0 ref 1 00000100:00100000:1.0:1789762832.370067:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067393536:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762832.370104:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c3c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.370117:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762832.370121:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762832.370126:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c3c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.370137:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c3c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.373831:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c3c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.373851:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c3c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.373860:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c3c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.373883:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762832.373889:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c3c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.373900:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c3c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.373957:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c3c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.373968:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175c3c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762832.374041:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067393536:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3979us (99356us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762832.374053:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 368 00000100:00100000:1.0:1789762832.374067:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305617038739 time 305617106574 state 0x2 ref 0 00000100:00100000:0.0:1789762832.376224:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762832.376267:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067393664 00000100:00100000:1.0:1789762832.376277:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067393664 00000100:00100000:1.0:1789762832.376322:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305617038739 time 305617106574 state 0x0 ref 1 00000100:00100000:1.0:1789762832.376333:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305717106574 time 305617106574 state 0x1 ref 1 00000020:00080000:3.0:1789762832.394220:0:7221:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 7 batchid updated 00000020:00080000:3.0:1789762832.394233:0:7221:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 8 batchid updated 00000004:00080000:3.0:1789762832.394248:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 13 committed 00000020:00080000:1.0:1789762832.394252:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 8 00000004:00080000:3.0:1789762832.394272:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 13 committed 00000004:00080000:1.0:1789762832.394315:0:7233:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c36c9a9dcc0 with version 18 00000020:00080000:1.0:1789762832.394333:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 7 cancel update log [0x200000401:0x2:0x0].7: rc = 0 00000100:00100000:2.0:1789762832.394352:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074677504:0@lo:6:osp-syn-0-1.0 00000100:00100000:3.0:1789762832.394355:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074677632:0@lo:6:osp-syn-1-1.0 00000100:00100000:2.0:1789762832.394410:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762832.394445:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074677504 00000100:00100000:3.0:1789762832.394453:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074677504 00000100:00100000:3.0:1789762832.394520:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:3.0:1789762832.394534:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074677504:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:3.0:1789762832.394558:0:8054:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:8 count 1 00002000:00010000:3.0:1789762832.394593:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x8:0x0] initial LVB size: 8388608, mtime: 0x6aad9d09, atime: 0x0, ctime: 0x6aad9d09, blocks: 0x4007 00010000:00010000:3.0:1789762832.394615:0:8054:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175c43 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.394631:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x8:0x0].0x0 since=305s v=(0 0) 00010000:00010000:3.0:1789762832.394636:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762832.394643:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175c43 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.394656:0:8054:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175c43 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.394666:0:8054:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175c43 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.394674:0:8054:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175c43 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.394692:0:8054:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762832.394708:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762832.394717:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175c43 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.394741:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762832.394746:0:7217:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175c43 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.394757:0:7217:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175c43 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.394774:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175c43 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.394780:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175c43 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.394799:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762832.394801:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762832.394803:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:1.0:1789762832.394944:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 7 cancel update log [0x240000400:0x2:0x0].7: rc = 0 00000100:00080000:2.0:1789762832.394951:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762832.394970:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1789762832.394979:0:7233:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c36c9a9d480 with version 19 00000004:00080000:0.0:1789762832.394987:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 18 rpc_version 18 00000020:00080000:1.0:1789762832.394988:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 8 cancel update log [0x200000401:0x2:0x0].8: rc = 0 00000020:00100000:0.0:1789762832.395010:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762832.395019:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37ff6bad80 x1876702074677888/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762832.395031:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074677632 00000100:00100000:0.0:1789762832.395036:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6bad80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074677888:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762832.395037:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074677632 00000100:00100000:2.0:1789762832.395081:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:0.0:1789762832.395090:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762832.395093:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074677632:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:2.0:1789762832.395111:0:9038:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:8 count 1 00000100:00100000:0.0:1789762832.395118:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c89766c0 going to sleep for 11 seconds 00002000:00010000:2.0:1789762832.395135:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0x8:0x0] initial LVB size: 8388608, mtime: 0x6aad9d09, atime: 0x0, ctime: 0x6aad9d09, blocks: 0x4007 00010000:00010000:2.0:1789762832.395142:0:9038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004f58f67a/0xdf5a7ac59f175c4a lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.395156:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x8:0x0].0x0 since=305s v=(0 0) 00010000:00010000:2.0:1789762832.395160:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762832.395163:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004f58f67a/0xdf5a7ac59f175c4a lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.395174:0:9038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004f58f67a/0xdf5a7ac59f175c4a lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.395180:0:9038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004f58f67a/0xdf5a7ac59f175c4a lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.395186:0:9038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004f58f67a/0xdf5a7ac59f175c4a lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.395200:0:9038:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000020:00080000:1.0:1789762832.395245:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 8 cancel update log [0x240000400:0x2:0x0].8: rc = 0 00000020:00080000:1.0:1789762832.395256:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 8 00000100:00100000:1.0:1789762832.395271:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074677888 00000100:00100000:1.0:1789762832.395276:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074677888 00000100:00100000:1.0:1789762832.395321:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1789762832.395330:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:7224:x1876702074677888:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1789762832.395339:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074677504:12345-0@lo:6:osp-syn-0-1.0 Request processed in 811us (930us total) trans 4294967792 rc 0/0 00000100:00100000:3.0:1789762832.395356:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000004:00080000:3.0:1789762832.395403:0:3424:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37fd9c1180/2, rc 0, transno 4294967792 00000100:00100000:3.0:1789762832.395410:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074677504:0@lo:6:osp-syn-0-1.0 00010000:00010000:2.0:1789762832.395663:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762832.395670:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004f58f67a/0xdf5a7ac59f175c4a lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.395680:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bd7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762832.395685:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004f58f67a/0xdf5a7ac59f175c4a lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.395695:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004f58f67a/0xdf5a7ac59f175c4a lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.395725:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004f58f67a/0xdf5a7ac59f175c4a lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.395735:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004f58f67a/0xdf5a7ac59f175c4a lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.395758:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762832.395762:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762832.395766:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762832.395773:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762832.395974:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074677888:12345-0@lo:1000:osp_up0-1.0 Request processed in 643us (892us total) trans 4294967396 rc 0/0 00000100:00100000:1.0:1789762832.395984:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00100000:0.0:1789762832.396005:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6bad80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074677888:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1789762832.396016:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 19 rpc_version 19 00000020:00100000:0.0:1789762832.396035:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762832.396039:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37ff6b9180 x1876702074678144/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762832.396050:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6b9180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074678144:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762832.396112:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762832.396133:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c89766c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789762832.396150:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074678144 00000100:00100000:0.0:1789762832.396154:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074678144 00000100:00100000:0.0:1789762832.396196:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283 00000100:00100000:0.0:1789762832.396208:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074678144:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762832.396272:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074677632:12345-0@lo:6:osp-syn-1-1.0 Request processed in 1182us (1303us total) trans 4294967381 rc 0/0 00000100:00100000:2.0:1789762832.396280:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:2.0:1789762832.396315:0:3425:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294967380 gen 2 00000004:00080000:2.0:1789762832.396319:0:3425:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37ff70df80, transno 4294967380 00000100:00100000:2.0:1789762832.396332:0:3425:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9c37f6088380 x1876702074677632/t4294967381(4294967381) o6->lustre-OST0001-osc-MDT0001@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789762848 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-1.0' uid:0 gid:0 projid:4294967295 00000004:00080000:2.0:1789762832.396342:0:3425:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37f6088380/2, rc 0, transno 4294967381 00000100:00100000:2.0:1789762832.396350:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074677632:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1789762832.396355:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7224:x1876702074678144:12345-0@lo:1000:osp_up0-1.0 Request processed in 149us (240us total) trans 4294967397 rc 0/0 00000100:00100000:0.0:1789762832.396363:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283 00000100:00100000:0.0:1789762832.396408:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6b9180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074678144:0@lo:1000:osp_up0-1.0 00000001:00080000:2.0:1789762832.397430:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967391 is committed 00000001:00080000:2.0:1789762832.397454:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967392 is committed 00000001:00080000:2.0:1789762832.397466:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967395 is committed 00010000:00010000:0.0:1789762832.397479:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175917 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.397488:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175917 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.397493:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175917 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.397497:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762832.397501:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175917 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.397507:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175917 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397508:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397532:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.397537:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x4009f790 (ffff9c37d3bee000) 00010000:00010000:0.0:1789762832.397545:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762832.397547:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.397549:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175917 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.397568:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175917 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397569:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762832.397577:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397587:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397621:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba100) 00010000:00010000:1.0:1789762832.397625:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762832.397629:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397637:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230e00/0xdf5a7ac59f175861 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397653:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f175868 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397659:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f175868 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397663:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f175868 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397666:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d92800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762832.397668:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f175868 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397672:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f175868 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397677:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:1.0:1789762832.397679:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762832.397681:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f175868 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397684:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92800/0xdf5a7ac59f175868 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397701:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f1758c3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397706:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f1758c3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397710:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f1758c3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397713:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762832.397715:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f1758c3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397719:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f1758c3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397723:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:1.0:1789762832.397725:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789762832.397727:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f1758c3 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397730:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a95000/0xdf5a7ac59f1758c3 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397736:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f1758e6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397740:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f1758e6 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397744:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f1758e6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397751:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a94c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762832.397753:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f1758e6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397757:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f1758e6 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397761:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:1.0:1789762832.397763:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789762832.397764:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f1758e6 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.397768:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1a94c00/0xdf5a7ac59f1758e6 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x12:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762832.470132:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305717106574 time 305717198347 state 0x0 ref 1 00000100:00100000:0.0:1789762832.470160:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067393664:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762832.470196:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c51 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.470210:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762832.470214:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762832.470219:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c51 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.470230:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c51 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.472997:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c51 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.473008:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c51 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.473022:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c51 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.473028:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762832.473031:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c51 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.473039:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c51 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.473053:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c51 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.473059:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c51 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762832.473107:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067393664:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2958us (96884us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762832.473114:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 369 00000100:00100000:0.0:1789762832.473122:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305717106574 time 305717198347 state 0x2 ref 0 00000100:00100000:0.0:1789762832.475278:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762832.475343:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067393792 00000100:00100000:1.0:1789762832.475353:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067393792 00000100:00100000:1.0:1789762832.475395:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305717106574 time 305717198347 state 0x0 ref 1 00000100:00100000:1.0:1789762832.475409:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305817198347 time 305717198347 state 0x1 ref 1 00000100:00100000:0.0:1789762832.570257:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305817198347 time 305817314449 state 0x0 ref 1 00000100:00100000:0.0:1789762832.570281:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067393792:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762832.570329:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c58 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.570343:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762832.570346:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762832.570351:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c58 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.570362:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c58 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.573299:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c58 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.573311:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c58 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.573318:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c58 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.573331:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762832.573335:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c58 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.573342:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c58 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.573357:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c58 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.573362:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f175c58 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762832.573411:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067393792:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3139us (98133us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762832.573418:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 370 00000100:00100000:1.0:1789762832.573425:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305817198347 time 305817314449 state 0x2 ref 0 00000100:00100000:2.0:1789762832.575517:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762832.575586:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067393920 00000100:00100000:1.0:1789762832.575594:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067393920 00000100:00100000:1.0:1789762832.575651:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305817198347 time 305817314449 state 0x0 ref 1 00000100:00100000:1.0:1789762832.575666:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305917314449 time 305817314449 state 0x1 ref 1 00000100:00100000:3.0:1789762832.670372:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305917314449 time 305917433842 state 0x0 ref 1 00000100:00100000:3.0:1789762832.670405:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067393920:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762832.670447:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c5f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.670462:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762832.670465:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762832.670470:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c5f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.670481:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c5f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.673288:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c5f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.673301:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c5f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.673308:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c5f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.673325:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762832.673329:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c5f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.673336:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c5f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.673349:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c5f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.673354:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c5f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762832.673402:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067393920:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3012us (97886us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762832.673409:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 371 00000100:00100000:3.0:1789762832.673417:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305917314449 time 305917433842 state 0x2 ref 0 00000100:00100000:2.0:1789762832.675646:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762832.675687:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067394048 00000100:00100000:0.0:1789762832.675697:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067394048 00000100:00100000:0.0:1789762832.675740:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 305917314449 time 305917433842 state 0x0 ref 1 00000100:00100000:0.0:1789762832.675751:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306017433842 time 305917433842 state 0x1 ref 1 00000100:00100000:0.0:1789762832.770454:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306017433842 time 306017518524 state 0x0 ref 1 00000100:00100000:0.0:1789762832.770482:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067394048:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762832.770518:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c66 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.770532:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762832.770536:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762832.770540:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c66 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.770566:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c66 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.773381:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c66 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.773396:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c66 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.773407:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c66 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.773415:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9e6a400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762832.773419:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c66 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.773426:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c66 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.773442:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c66 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.773448:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c66 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762832.773507:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067394048:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3036us (97861us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762832.773516:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 372 00000100:00100000:0.0:1789762832.773524:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306017433842 time 306017518524 state 0x2 ref 0 00000100:00100000:0.0:1789762832.775341:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762832.775389:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067394176 00000100:00100000:1.0:1789762832.775402:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067394176 00000100:00100000:1.0:1789762832.775436:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306017433842 time 306017518524 state 0x0 ref 1 00000100:00100000:1.0:1789762832.775447:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306117518524 time 306017518524 state 0x1 ref 1 00000100:00100000:2.0:1789762832.839744:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762832.839779:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762832.839803:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067394304 00000100:00100000:0.0:1789762832.839812:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067394304 00000100:00100000:2.0:1789762832.839831:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762832.839832:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067394560 00000100:00100000:3.0:1789762832.839839:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067394560 00000100:00100000:0.0:1789762832.839859:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 345 00000100:00100000:0.0:1789762832.839870:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d157b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067394304:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762832.839878:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762832.839891:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067394816 00000100:00100000:3.0:1789762832.839895:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067394816 00000100:00100000:1.0:1789762832.839905:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067394432 00000100:00100000:1.0:1789762832.839913:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067394432 00000100:00100000:3.0:1789762832.839948:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 418 00000100:00100000:0.0:1789762832.839954:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d157b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067394304:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 84us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762832.839961:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762832.839964:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067394560:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762832.839971:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 345 00000100:00100000:1.0:1789762832.839994:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 419 00000100:00100000:3.0:1789762832.840005:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067394560:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 45us (225us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762832.840009:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067394816:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762832.840016:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 420 00000100:00100000:3.0:1789762832.840028:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 418 00000100:00100000:0.0:1789762832.840032:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2398:x1876702067394432:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762832.840065:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067394688 00000100:00100000:1.0:1789762832.840066:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067394816:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 58us (232us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762832.840070:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762832.840072:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2398:x1876702067394432:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 42us (198us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762832.840072:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067394688 00000100:00100000:1.0:1789762832.840088:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 419 00000100:00100000:0.0:1789762832.840096:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 420 00000100:00100000:1.0:1789762832.840115:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067394944 00000100:00100000:1.0:1789762832.840121:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067394944 00000100:00100000:3.0:1789762832.840126:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 162 00000100:00100000:3.0:1789762832.840138:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067394688:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762832.840168:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 421 00000100:00100000:3.0:1789762832.840174:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067394688:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 36us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762832.840182:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 162 00000100:00100000:1.0:1789762832.840188:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067394944:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:2.0:1789762832.840206:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762832.840212:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762832.840216:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762832.840240:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067394944:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 60us (178us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762832.840248:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 421 00000100:00100000:0.0:1789762832.870577:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306117518524 time 306117642049 state 0x0 ref 1 00000100:00100000:0.0:1789762832.870738:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067394176:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762832.870776:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c6d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.870802:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762832.870806:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762832.870811:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c6d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.870822:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c6d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.873735:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c6d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.873746:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c6d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.873752:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c6d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.873764:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9e6a400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762832.873767:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c6d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.873774:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c6d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.873787:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c6d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.873792:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b72603e/0xdf5a7ac59f175c6d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762832.873841:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067394176:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3249us (98501us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762832.873848:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 373 00000100:00100000:1.0:1789762832.873855:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306117518524 time 306117642049 state 0x2 ref 0 00000100:00100000:2.0:1789762832.876002:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762832.876047:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067395072 00000100:00100000:3.0:1789762832.876055:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067395072 00000100:00100000:3.0:1789762832.876091:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306117518524 time 306117642049 state 0x0 ref 1 00000100:00100000:3.0:1789762832.876102:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306217642049 time 306117642049 state 0x1 ref 1 00000100:00100000:1.0:1789762832.970692:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306217642049 time 306217754784 state 0x0 ref 1 00000100:00100000:1.0:1789762832.970718:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067395072:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762832.970759:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c74 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.970795:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762832.970799:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762832.970806:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c74 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.970820:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c74 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.973874:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c74 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.973888:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c74 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.973900:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c74 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.973908:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762832.973913:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c74 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.973950:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c74 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.973969:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c74 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.973989:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c74 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762832.974046:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067395072:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3335us (98045us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762832.974055:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 374 00000100:00100000:1.0:1789762832.974066:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306217642049 time 306217754784 state 0x2 ref 0 00000100:00100000:2.0:1789762832.976066:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762832.976116:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067395200 00000100:00100000:0.0:1789762832.976125:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067395200 00000100:00100000:0.0:1789762832.976166:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306217642049 time 306217754784 state 0x0 ref 1 00000100:00100000:0.0:1789762832.976180:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306317754784 time 306217754784 state 0x1 ref 1 00010000:00010000:1.0:1789762833.024094:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762833.070788:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306317754784 time 306317849741 state 0x0 ref 1 00000100:00100000:0.0:1789762833.070838:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067395200:12345-192.168.203.37@tcp:3:dd.0 00000100:00080000:2.0:1789762833.070876:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762833.070882:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c7b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.070899:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762833.070905:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762833.070912:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c7b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.070974:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c7b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.073655:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c7b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.073666:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c7b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.073673:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c7b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.073679:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762833.073683:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c7b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.073689:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c7b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.073701:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c7b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.073705:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c7b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762833.073760:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067395200:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2954us (97695us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762833.073767:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 375 00000100:00100000:0.0:1789762833.073775:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306317754784 time 306317849741 state 0x2 ref 0 00000100:00100000:0.0:1789762833.075649:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762833.075715:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067395328 00000100:00100000:1.0:1789762833.075724:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067395328 00000100:00100000:1.0:1789762833.075766:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306317754784 time 306317849741 state 0x0 ref 1 00000100:00100000:1.0:1789762833.075781:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306417849741 time 306317849741 state 0x1 ref 1 00000100:00100000:0.0:1789762833.170865:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306417849741 time 306417931117 state 0x0 ref 1 00000100:00100000:0.0:1789762833.170890:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067395328:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762833.170955:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c82 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.170970:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762833.170974:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762833.170979:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c82 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.170997:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c82 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.174001:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c82 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.174017:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c82 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.174026:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c82 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.174035:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762833.174051:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c82 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.174061:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c82 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.174082:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c82 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.174092:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f175c82 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762833.174156:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067395328:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3274us (98504us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762833.174168:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 376 00000100:00100000:1.0:1789762833.174180:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306417849741 time 306417931117 state 0x2 ref 0 00000100:00100000:2.0:1789762833.176205:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762833.176248:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067395456 00000100:00100000:1.0:1789762833.176258:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067395456 00000100:00100000:1.0:1789762833.176299:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306417849741 time 306417931117 state 0x0 ref 1 00000100:00100000:1.0:1789762833.176313:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306517931117 time 306417931117 state 0x1 ref 1 00000100:00100000:1.0:1789762833.270963:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306517931117 time 306518025309 state 0x0 ref 1 00000100:00100000:1.0:1789762833.270992:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067395456:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762833.271045:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c89 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.271062:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762833.271066:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762833.271072:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c89 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.271086:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c89 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.273963:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c89 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.273974:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c89 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.273983:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c89 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.273990:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762833.273994:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c89 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.274004:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c89 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.274034:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c89 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.274042:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c89 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762833.274098:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067395456:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3117us (97893us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762833.274108:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 377 00000100:00100000:1.0:1789762833.274119:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306517931117 time 306518025309 state 0x2 ref 0 00000100:00100000:2.0:1789762833.275914:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762833.276011:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067395584 00000100:00100000:0.0:1789762833.276020:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067395584 00000100:00100000:0.0:1789762833.276057:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306517931117 time 306518025309 state 0x0 ref 1 00000100:00100000:0.0:1789762833.276078:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306618025309 time 306518025309 state 0x1 ref 1 00000100:00100000:3.0:1789762833.371099:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306618025309 time 306618165037 state 0x0 ref 1 00000100:00100000:3.0:1789762833.371126:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067395584:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762833.371163:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c90 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.371176:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762833.371180:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762833.371185:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c90 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.371212:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c90 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.373986:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c90 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.373997:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c90 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.374004:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c90 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.374010:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762833.374013:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c90 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.374020:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c90 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.374033:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c90 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.374038:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175c90 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762833.374103:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067395584:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2986us (98189us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762833.374111:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 378 00000100:00100000:3.0:1789762833.374119:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306618025309 time 306618165037 state 0x2 ref 0 00000100:00100000:0.0:1789762833.375975:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762833.376043:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067395712 00000100:00100000:1.0:1789762833.376052:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067395712 00000100:00100000:1.0:1789762833.376093:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306618025309 time 306618165037 state 0x0 ref 1 00000100:00100000:1.0:1789762833.376107:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306718165037 time 306618165037 state 0x1 ref 1 00000100:00100000:0.0:1789762833.471201:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306718165037 time 306718266798 state 0x0 ref 1 00000100:00100000:0.0:1789762833.471227:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067395712:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762833.471265:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175c97 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.471279:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762833.471283:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762833.471287:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175c97 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.471298:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175c97 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.474139:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175c97 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.474149:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175c97 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.474155:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175c97 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.474175:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5870600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762833.474180:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175c97 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.474190:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175c97 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.474210:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175c97 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.474218:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175c97 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762833.474272:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067395712:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3055us (98298us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762833.474280:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 379 00000100:00100000:0.0:1789762833.474289:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306718165037 time 306718266798 state 0x2 ref 0 00000100:00100000:0.0:1789762833.476288:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762833.476354:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067395840 00000100:00100000:1.0:1789762833.476362:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067395840 00000100:00100000:1.0:1789762833.476406:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306718165037 time 306718266798 state 0x0 ref 1 00000100:00100000:1.0:1789762833.476420:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306818266798 time 306718266798 state 0x1 ref 1 00000100:00100000:1.0:1789762833.571293:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306818266798 time 306818353738 state 0x0 ref 1 00000100:00100000:1.0:1789762833.571347:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067395840:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762833.571386:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c9e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.571402:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762833.571408:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762833.571414:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c9e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.571428:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c9e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.574405:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c9e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.574416:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c9e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.574424:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c9e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.574430:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762833.574434:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c9e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.574440:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c9e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.574463:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c9e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.574469:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175c9e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762833.574521:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067395840:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3183us (98233us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762833.574528:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 380 00000100:00100000:1.0:1789762833.574537:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306818266798 time 306818353738 state 0x2 ref 0 00000100:00100000:0.0:1789762833.576294:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762833.576334:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067395968 00000100:00100000:3.0:1789762833.576342:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067395968 00000100:00100000:3.0:1789762833.576377:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306818266798 time 306818353738 state 0x0 ref 1 00000100:00100000:3.0:1789762833.576387:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306918353738 time 306818353738 state 0x1 ref 1 00000100:00100000:0.0:1789762833.671407:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306918353738 time 306918473524 state 0x0 ref 1 00000100:00100000:0.0:1789762833.671434:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067395968:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762833.671471:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175ca5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.671499:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762833.671502:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762833.671507:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175ca5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.671518:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175ca5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.674162:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175ca5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.674176:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175ca5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.674185:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175ca5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.674193:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5870600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762833.674199:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175ca5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.674209:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175ca5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.674236:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175ca5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.674245:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175ca5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762833.674301:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067395968:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2877us (98007us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762833.674311:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 381 00000100:00100000:1.0:1789762833.674321:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306918353738 time 306918473524 state 0x2 ref 0 00000100:00100000:2.0:1789762833.676155:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762833.676233:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067396096 00000100:00100000:0.0:1789762833.676242:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067396096 00000100:00100000:0.0:1789762833.676279:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 306918353738 time 306918473524 state 0x0 ref 1 00000100:00100000:0.0:1789762833.676290:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307018473524 time 306918473524 state 0x1 ref 1 00000100:00100000:0.0:1789762833.771473:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307018473524 time 307018539069 state 0x0 ref 1 00000100:00100000:0.0:1789762833.771498:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067396096:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762833.771534:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f175cac lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.771548:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762833.771552:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762833.771556:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f175cac lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.771585:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f175cac lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.774411:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f175cac lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.774422:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f175cac lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.774428:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f175cac lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.774433:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5871000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762833.774437:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f175cac lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.774443:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f175cac lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.774464:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f175cac lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.774470:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005782f968/0xdf5a7ac59f175cac lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762833.774521:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067396096:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3032us (98365us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762833.774528:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 382 00000100:00100000:1.0:1789762833.774537:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307018473524 time 307018539069 state 0x2 ref 0 00000100:00100000:2.0:1789762833.776343:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762833.776378:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067396224 00000100:00100000:1.0:1789762833.776384:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067396224 00000100:00100000:1.0:1789762833.776419:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307018473524 time 307018539069 state 0x0 ref 1 00000100:00100000:1.0:1789762833.776428:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307118539069 time 307018539069 state 0x1 ref 1 00000100:00100000:1.0:1789762833.871573:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307118539069 time 307118634494 state 0x0 ref 1 00000100:00100000:1.0:1789762833.871617:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067396224:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762833.871657:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cb3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.871673:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762833.871691:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762833.871698:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cb3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.871712:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cb3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.874713:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cb3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.874726:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cb3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.874736:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cb3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.874745:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762833.874750:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cb3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.874759:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cb3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.874777:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cb3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.874785:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cb3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762833.874845:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067396224:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3254us (98502us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762833.874856:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 383 00000100:00100000:1.0:1789762833.874866:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307118539069 time 307118634494 state 0x2 ref 0 00000100:00100000:0.0:1789762833.876997:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762833.877032:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067396352 00000100:00100000:1.0:1789762833.877040:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067396352 00000100:00100000:1.0:1789762833.877079:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307118539069 time 307118634494 state 0x0 ref 1 00000100:00100000:1.0:1789762833.877092:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307218634494 time 307118634494 state 0x1 ref 1 00000100:00100000:3.0:1789762833.920049:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074678272:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762833.920079:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ff8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074678656:0@lo:400:kworker.0 00000100:00100000:3.0:1789762833.920121:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762833.920134:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762833.920171:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074678272 00000100:00100000:0.0:1789762833.920179:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074678272 00000100:00100000:2.0:1789762833.920190:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074678400:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762833.920206:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ff9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074679168:0@lo:400:kworker.0 00000100:00100000:3.0:1789762833.920207:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ff8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074678784:0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.920224:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762833.920237:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074678400 00000100:00100000:0.0:1789762833.920242:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074678400 00000100:00100000:2.0:1789762833.920253:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074678912:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762833.920263:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762833.920273:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:3.0:1789762833.920274:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff70df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074678528:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762833.920283:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762833.920286:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074678272:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762833.920303:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9afb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074679680:0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.920312:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ff9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074679424:0@lo:400:kworker.0 00002000:00000024:0.0:1789762833.920318:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 13631488 granted 71368704 pending 0 free 3851419648 avail 3849322496 00000100:00100000:3.0:1789762833.920326:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762833.920349:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762833.920356:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074679040:0@lo:400:kworker.0 00000100:00100000:1.0:1789762833.920367:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074678656 00000100:00100000:1.0:1789762833.920374:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074678656 00000100:00100000:2.0:1789762833.920384:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074679936:0@lo:400:kworker.0 00000100:00100000:0.0:1789762833.920385:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074678272:12345-0@lo:13:osp-pre-1-1.0 Request processed in 101us (265us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762833.920398:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00100000:3.0:1789762833.920403:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762833.920419:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762833.920422:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074678912 00000100:00100000:1.0:1789762833.920423:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 346 00000100:00100000:0.0:1789762833.920431:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074678912 00000100:00100000:3.0:1789762833.920438:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9af480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074679552:0@lo:400:kworker.0 00000100:00100000:1.0:1789762833.920439:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1578450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074678656:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.920463:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074679168 00000100:00100000:2.0:1789762833.920468:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074679168 00000100:00100000:0.0:1789762833.920478:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074678528 00000100:00100000:0.0:1789762833.920486:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074678528 00000100:00100000:3.0:1789762833.920486:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762833.920495:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1578450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074678656:12345-0@lo:400:kworker.0 Request processed in 61us (363us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762833.920504:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 346 00000100:00100000:2.0:1789762833.920509:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074679424 00000100:00100000:2.0:1789762833.920513:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074679424 00000100:00100000:0.0:1789762833.920522:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:3.0:1789762833.920525:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9aed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074680064:0@lo:400:kworker.0 00000100:00100000:0.0:1789762833.920536:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074678400:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762833.920542:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074679552 00000100:00100000:2.0:1789762833.920545:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074679552 00002000:00000024:0.0:1789762833.920560:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439418880 pending 0 free 3746562048 avail 3744464896 00000100:00100000:1.0:1789762833.920561:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:3.0:1789762833.920569:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762833.920582:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074678912:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762833.920582:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 422 00000100:00100000:2.0:1789762833.920610:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074679168:12345-0@lo:400:kworker.0 00002000:00000024:1.0:1789762833.920615:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439418880 pending 0 free 3746562048 avail 3744464896 00000100:00100000:0.0:1789762833.920629:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074678400:12345-0@lo:13:osp-pre-0-1.0 Request processed in 96us (403us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762833.920636:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00100000:3.0:1789762833.920650:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074679936 00000100:00100000:0.0:1789762833.920656:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:3.0:1789762833.920657:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074679936 00000100:00100000:2.0:1789762833.920663:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074679168:12345-0@lo:400:kworker.0 Request processed in 70us (399us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762833.920668:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee760a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074678528:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762833.920676:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074678912:12345-0@lo:13:osp-pre-0-0.0 Request processed in 99us (393us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762833.920676:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 422 00000100:00100000:1.0:1789762833.920691:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00002000:00000024:0.0:1789762833.920692:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 13631488 granted 71368704 pending 0 free 3851419648 avail 3849322496 00000100:00100000:3.0:1789762833.920697:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074680064 00000100:00100000:3.0:1789762833.920702:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074680064 00000100:00100000:2.0:1789762833.920703:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 423 00000100:00100000:2.0:1789762833.920715:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074679424:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762833.920729:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762833.920738:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee760a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074678528:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (415us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762833.920749:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:3.0:1789762833.920750:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:1.0:1789762833.920757:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9ac700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074680192:0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.920759:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074679424:12345-0@lo:400:kworker.0 Request processed in 46us (409us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762833.920768:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074679936:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.920774:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 423 00000100:00080000:0.0:1789762833.920778:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762833.920784:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762833.920788:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762833.920791:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074679680 00000100:00080000:0.0:1789762833.920792:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762833.920796:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074679680 00000100:00100000:1.0:1789762833.920801:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789762833.920806:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762833.920831:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 424 00000100:00100000:3.0:1789762833.920832:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074679936:12345-0@lo:400:kworker.0 Request processed in 68us (412us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762833.920845:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074679552:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.920847:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:3.0:1789762833.920848:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:1.0:1789762833.920849:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ff8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074678656:0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.920867:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074679680:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762833.920875:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074680192 00000100:00100000:3.0:1789762833.920881:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074680192 00000100:00100000:0.0:1789762833.920892:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074679552:12345-0@lo:400:kworker.0 Request processed in 52us (408us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762833.920893:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ff9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074679168:0@lo:400:kworker.0 00000100:00100000:0.0:1789762833.920903:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 424 00000100:00100000:3.0:1789762833.920912:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:2.0:1789762833.920943:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074679680:12345-0@lo:400:kworker.0 Request processed in 81us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762833.920945:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9afb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074679680:0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.920955:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00100000:3.0:1789762833.920955:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074680064:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762833.920977:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074678400:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762833.920994:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074678912:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762833.921010:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074680064:12345-0@lo:400:kworker.0 Request processed in 60us (445us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762833.921012:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ff9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074679424:0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.921014:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:1.0:1789762833.921020:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074679040 00000100:00100000:0.0:1789762833.921028:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074679936:0@lo:400:kworker.0 00000100:00100000:1.0:1789762833.921028:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074679040 00000100:00100000:3.0:1789762833.921035:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00100000:2.0:1789762833.921036:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074680192:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762833.921072:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:1.0:1789762833.921085:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074679040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.921090:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074680192:12345-0@lo:400:kworker.0 Request processed in 61us (288us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762833.921094:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762833.921103:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:3.0:1789762833.921116:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff70df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074678528:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762833.921125:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074679296:0@lo:400:kworker.0 00000100:00100000:3.0:1789762833.921139:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9af480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074679552:0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.921144:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074678784 00000100:00100000:2.0:1789762833.921149:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074678784 00000100:00100000:1.0:1789762833.921156:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074679040:12345-0@lo:400:kworker.0 Request processed in 72us (756us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762833.921162:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9aed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074680064:0@lo:400:kworker.0 00000100:00100000:0.0:1789762833.921165:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762833.921174:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00100000:2.0:1789762833.921179:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 426 00000100:00100000:2.0:1789762833.921189:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074678784:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762833.921200:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074679296 00000100:00100000:0.0:1789762833.921206:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9aea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074679808:0@lo:400:kworker.0 00000100:00100000:1.0:1789762833.921207:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074679296 00000100:00100000:3.0:1789762833.921211:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074679040:0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.921236:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074678784:12345-0@lo:400:kworker.0 Request processed in 50us (147us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762833.921243:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 426 00000100:00100000:1.0:1789762833.921251:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:0.0:1789762833.921256:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762833.921268:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074679296:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.921290:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074679808 00000100:00100000:0.0:1789762833.921291:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074680320:0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.921295:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074679808 00000100:00100000:1.0:1789762833.921311:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074679296:12345-0@lo:400:kworker.0 Request processed in 48us (151us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762833.921334:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00000100:00100000:2.0:1789762833.921335:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 427 00000100:00100000:0.0:1789762833.921344:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762833.921349:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074679808:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789762833.921352:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762833.921374:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074680320 00000100:00100000:3.0:1789762833.921379:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074680320 00000100:00100000:0.0:1789762833.921390:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074678272:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762833.921394:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9ac700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074680192:0@lo:400:kworker.0 00000100:00100000:2.0:1789762833.921395:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074679808:12345-0@lo:400:kworker.0 Request processed in 47us (145us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762833.921409:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 427 00000100:00100000:3.0:1789762833.921419:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:0.0:1789762833.921426:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ff8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074678784:0@lo:400:kworker.0 00000100:00100000:3.0:1789762833.921433:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074680320:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762833.921442:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074679296:0@lo:400:kworker.0 00000100:00100000:0.0:1789762833.921453:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9aea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074679808:0@lo:400:kworker.0 00000100:00100000:0.0:1789762833.921477:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074680320:0@lo:400:kworker.0 00000100:00100000:3.0:1789762833.921484:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074680320:12345-0@lo:400:kworker.0 Request processed in 54us (143us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762833.921499:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:1.0:1789762833.971645:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307218634494 time 307218707208 state 0x0 ref 1 00000100:00100000:1.0:1789762833.971668:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067396352:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762833.971705:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cba lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.971722:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762833.971727:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762833.971733:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cba lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.971747:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cba lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.974473:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cba lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.974484:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cba lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.974493:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cba lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.974516:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762833.974521:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cba lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.974530:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cba lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.974547:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cba lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.974556:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175cba lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762833.974624:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067396352:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2961us (97626us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762833.974634:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 384 00000100:00100000:1.0:1789762833.974644:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307218634494 time 307218707208 state 0x2 ref 0 00000100:00100000:0.0:1789762833.976735:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762833.976776:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067396480 00000100:00100000:1.0:1789762833.976784:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067396480 00000100:00100000:1.0:1789762833.976822:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307218634494 time 307218707208 state 0x0 ref 1 00000100:00100000:1.0:1789762833.976835:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307318707208 time 307218707208 state 0x1 ref 1 00010000:00010000:3.0:1789762834.048063:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762834.071745:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307318707208 time 307318809873 state 0x0 ref 1 00000100:00100000:3.0:1789762834.071772:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067396480:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762834.071809:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cc1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.071823:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762834.071827:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762834.071831:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cc1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.071841:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cc1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.074831:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cc1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.074841:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cc1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.074848:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cc1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.074853:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762834.074867:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cc1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.074875:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cc1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.074890:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cc1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.074899:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cc1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762834.074983:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067396480:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3221us (98247us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762834.074991:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 385 00000100:00100000:3.0:1789762834.074999:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307318707208 time 307318809873 state 0x2 ref 0 00000100:00100000:0.0:1789762834.076884:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762834.076941:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067396608 00000100:00100000:1.0:1789762834.076955:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067396608 00000100:00100000:1.0:1789762834.076995:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307318707208 time 307318809873 state 0x0 ref 1 00000100:00100000:1.0:1789762834.077009:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307418809873 time 307318809873 state 0x1 ref 1 00000100:00100000:0.0:1789762834.171836:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307418809873 time 307418901478 state 0x0 ref 1 00000100:00100000:0.0:1789762834.171861:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067396608:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762834.171900:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175cc8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.171915:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.171935:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.171939:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175cc8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.171951:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175cc8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.174644:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175cc8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.174666:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175cc8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.174673:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175cc8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.174678:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762834.174681:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175cc8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.174687:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175cc8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.174700:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175cc8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.174705:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175cc8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762834.174751:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067396608:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2899us (97867us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762834.174758:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 386 00000100:00100000:0.0:1789762834.174766:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307418809873 time 307418901478 state 0x2 ref 0 00000100:00100000:0.0:1789762834.176555:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762834.176612:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067396736 00000100:00100000:1.0:1789762834.176621:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067396736 00000100:00100000:1.0:1789762834.176661:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307418809873 time 307418901478 state 0x0 ref 1 00000100:00100000:1.0:1789762834.176675:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307518901478 time 307418901478 state 0x1 ref 1 00000100:00100000:0.0:1789762834.271972:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307518901478 time 307519035884 state 0x0 ref 1 00000100:00100000:0.0:1789762834.271997:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067396736:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762834.272033:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175ccf lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.272047:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.272051:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.272056:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175ccf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.272066:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175ccf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.274988:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175ccf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.275003:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175ccf lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.275013:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175ccf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.275021:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762834.275026:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175ccf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.275036:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175ccf lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.275072:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175ccf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.275080:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a01a9da/0xdf5a7ac59f175ccf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762834.275134:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067396736:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3146us (98579us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762834.275143:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 387 00000100:00100000:1.0:1789762834.275153:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307518901478 time 307519035884 state 0x2 ref 0 00000100:00100000:2.0:1789762834.277007:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762834.277042:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067396864 00000100:00100000:3.0:1789762834.277051:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067396864 00000100:00100000:3.0:1789762834.277087:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307518901478 time 307519035884 state 0x0 ref 1 00000100:00100000:3.0:1789762834.277097:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307619035884 time 307519035884 state 0x1 ref 1 00000100:00100000:0.0:1789762834.372069:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307619035884 time 307619134938 state 0x0 ref 1 00000100:00100000:0.0:1789762834.372095:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067396864:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762834.372131:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cd6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.372144:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.372148:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.372153:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cd6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.372164:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cd6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.374668:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cd6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.374679:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cd6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.374685:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cd6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.374691:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762834.374694:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cd6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.374700:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cd6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.374713:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cd6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.374718:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cd6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762834.374769:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067396864:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2685us (97762us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762834.374777:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 388 00000100:00100000:0.0:1789762834.374792:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307619035884 time 307619134938 state 0x2 ref 0 00000100:00100000:0.0:1789762834.376708:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762834.376770:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067396992 00000100:00100000:1.0:1789762834.376780:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067396992 00000100:00100000:1.0:1789762834.376822:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307619035884 time 307619134938 state 0x0 ref 1 00000100:00100000:1.0:1789762834.376836:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307719134938 time 307619134938 state 0x1 ref 1 00000100:00100000:0.0:1789762834.472178:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307719134938 time 307719242573 state 0x0 ref 1 00000100:00100000:0.0:1789762834.472204:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2528000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067396992:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762834.472242:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cdd lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.472255:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.472258:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.472263:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cdd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.472273:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cdd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.475371:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cdd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.475383:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cdd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.475390:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cdd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.475407:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff935200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762834.475411:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cdd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.475418:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cdd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.475432:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cdd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.475438:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de312eb5/0xdf5a7ac59f175cdd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762834.475488:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2528000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067396992:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3295us (98780us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762834.475497:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 389 00000100:00100000:1.0:1789762834.475508:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307719134938 time 307719242573 state 0x2 ref 0 00000100:00100000:2.0:1789762834.477378:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762834.477419:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067397120 00000100:00100000:1.0:1789762834.477428:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067397120 00000100:00100000:1.0:1789762834.477463:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307719134938 time 307719242573 state 0x0 ref 1 00000100:00100000:1.0:1789762834.477474:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307819242573 time 307719242573 state 0x1 ref 1 00000100:00100000:1.0:1789762834.572272:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307819242573 time 307819336164 state 0x0 ref 1 00000100:00100000:1.0:1789762834.572311:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067397120:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762834.572354:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175ce4 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.572372:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762834.572376:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762834.572382:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175ce4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.572393:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175ce4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.575678:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175ce4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.575690:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175ce4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.575697:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175ce4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.575703:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762834.575706:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175ce4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.575713:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175ce4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.575743:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175ce4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.575752:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000816af118/0xdf5a7ac59f175ce4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762834.575805:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067397120:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3502us (98426us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762834.575812:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 390 00000100:00100000:1.0:1789762834.575821:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307819242573 time 307819336164 state 0x2 ref 0 00000100:00100000:0.0:1789762834.577738:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762834.577777:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067397248 00000100:00100000:1.0:1789762834.577784:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067397248 00000100:00100000:1.0:1789762834.577820:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307819242573 time 307819336164 state 0x0 ref 1 00000100:00100000:1.0:1789762834.577830:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307919336164 time 307819336164 state 0x1 ref 1 00000100:00100000:0.0:1789762834.672401:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307919336164 time 307919464889 state 0x0 ref 1 00000100:00100000:0.0:1789762834.672428:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067397248:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762834.672465:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007a13773e/0xdf5a7ac59f175ceb lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.672479:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.672482:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.672500:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007a13773e/0xdf5a7ac59f175ceb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.672512:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007a13773e/0xdf5a7ac59f175ceb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.675125:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007a13773e/0xdf5a7ac59f175ceb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.675137:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007a13773e/0xdf5a7ac59f175ceb lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.675144:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007a13773e/0xdf5a7ac59f175ceb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.675161:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bd3400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762834.675165:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007a13773e/0xdf5a7ac59f175ceb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.675172:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007a13773e/0xdf5a7ac59f175ceb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.675187:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007a13773e/0xdf5a7ac59f175ceb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.675192:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007a13773e/0xdf5a7ac59f175ceb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762834.675240:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067397248:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2823us (97502us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762834.675247:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 391 00000100:00100000:1.0:1789762834.675255:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307919336164 time 307919464889 state 0x2 ref 0 00000100:00100000:0.0:1789762834.677085:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762834.677120:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067397376 00000100:00100000:1.0:1789762834.677128:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067397376 00000100:00100000:1.0:1789762834.677162:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 307919336164 time 307919464889 state 0x0 ref 1 00000100:00100000:1.0:1789762834.677173:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308019464889 time 307919464889 state 0x1 ref 1 00000100:00100000:3.0:1789762834.772494:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308019464889 time 308019560221 state 0x0 ref 1 00000100:00100000:3.0:1789762834.772519:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067397376:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762834.772569:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cf2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.772596:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762834.772602:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762834.772607:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cf2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.772620:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cf2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.775562:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cf2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.775574:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cf2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.775580:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cf2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.775600:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762834.775604:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cf2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.775611:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cf2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.775635:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cf2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762834.775640:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175cf2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762834.775687:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067397376:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3179us (98603us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762834.775695:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 392 00000100:00100000:3.0:1789762834.775703:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308019464889 time 308019560221 state 0x2 ref 0 00000100:00100000:2.0:1789762834.777592:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762834.777636:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067397504 00000100:00100000:1.0:1789762834.777651:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067397504 00000100:00100000:1.0:1789762834.777686:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308019464889 time 308019560221 state 0x0 ref 1 00000100:00100000:1.0:1789762834.777697:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308119560221 time 308019560221 state 0x1 ref 1 00000100:00100000:0.0:1789762834.872685:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308119560221 time 308119749438 state 0x0 ref 1 00000100:00100000:0.0:1789762834.872713:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067397504:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762834.872749:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a1ba37d/0xdf5a7ac59f175cf9 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.872763:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.872767:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.872772:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a1ba37d/0xdf5a7ac59f175cf9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.872783:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a1ba37d/0xdf5a7ac59f175cf9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.875833:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a1ba37d/0xdf5a7ac59f175cf9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.875848:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a1ba37d/0xdf5a7ac59f175cf9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.875859:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a1ba37d/0xdf5a7ac59f175cf9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.875868:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d989b000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762834.875874:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a1ba37d/0xdf5a7ac59f175cf9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.875884:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a1ba37d/0xdf5a7ac59f175cf9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.875902:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a1ba37d/0xdf5a7ac59f175cf9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.875910:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a1ba37d/0xdf5a7ac59f175cf9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762834.876018:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067397504:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3314us (98435us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762834.876030:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 393 00000100:00100000:1.0:1789762834.876041:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308119560221 time 308119749438 state 0x2 ref 0 00000100:00100000:2.0:1789762834.877945:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762834.877977:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067397632 00000100:00100000:1.0:1789762834.877987:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067397632 00000100:00100000:1.0:1789762834.878030:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308119560221 time 308119749438 state 0x0 ref 1 00000100:00100000:1.0:1789762834.878044:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308219749438 time 308119749438 state 0x1 ref 1 00000100:00100000:0.0:1789762834.972790:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308219749438 time 308219856243 state 0x0 ref 1 00000100:00100000:0.0:1789762834.972813:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067397632:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762834.972848:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d020dd89/0xdf5a7ac59f175d00 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.972874:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.972877:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.972882:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d020dd89/0xdf5a7ac59f175d00 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.972893:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d020dd89/0xdf5a7ac59f175d00 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.975631:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d020dd89/0xdf5a7ac59f175d00 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.975642:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d020dd89/0xdf5a7ac59f175d00 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.975649:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d020dd89/0xdf5a7ac59f175d00 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.975654:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d989a000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762834.975657:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d020dd89/0xdf5a7ac59f175d00 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.975663:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d020dd89/0xdf5a7ac59f175d00 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.975685:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d020dd89/0xdf5a7ac59f175d00 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.975691:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d020dd89/0xdf5a7ac59f175d00 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762834.975739:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067397632:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2934us (97796us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762834.975746:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 394 00000100:00100000:1.0:1789762834.975754:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308219749438 time 308219856243 state 0x2 ref 0 00000100:00100000:2.0:1789762834.977632:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762834.977671:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067397760 00000100:00100000:3.0:1789762834.977678:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067397760 00000100:00100000:3.0:1789762834.977713:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308219749438 time 308219856243 state 0x0 ref 1 00000100:00100000:3.0:1789762834.977722:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308319856243 time 308219856243 state 0x1 ref 1 00010000:00010000:2.0:1789762835.072042:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762835.072863:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308319856243 time 308319930500 state 0x0 ref 1 00000100:00100000:0.0:1789762835.072904:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067397760:12345-192.168.203.37@tcp:3:dd.0 00000100:00080000:3.0:1789762835.072939:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762835.072966:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d07 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.072980:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762835.072984:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762835.072988:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d07 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.073004:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d07 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.075968:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d07 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.076001:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d07 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.076008:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d07 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.076014:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d989be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762835.076018:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d07 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.076025:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d07 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.076040:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d07 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.076045:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d07 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762835.076099:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067397760:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3221us (98467us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762835.076107:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 395 00000100:00100000:0.0:1789762835.076115:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308319856243 time 308319930500 state 0x2 ref 0 00000100:00100000:0.0:1789762835.078026:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762835.078076:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067397888 00000100:00100000:1.0:1789762835.078084:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067397888 00000100:00100000:1.0:1789762835.078124:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308319856243 time 308319930500 state 0x0 ref 1 00000100:00100000:1.0:1789762835.078135:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308419930500 time 308319930500 state 0x1 ref 1 00000100:00100000:0.0:1789762835.172965:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308419930500 time 308420030055 state 0x0 ref 1 00000100:00100000:0.0:1789762835.172996:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067397888:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762835.173032:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d0e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.173045:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762835.173049:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762835.173054:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d0e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.173064:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d0e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.175938:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d0e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.175949:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d0e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.175956:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d0e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.175985:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d989be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762835.175990:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d0e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.176000:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d0e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.176019:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d0e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.176028:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000059971b2c/0xdf5a7ac59f175d0e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762835.176078:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067397888:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3097us (98053us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762835.176084:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 396 00000100:00100000:1.0:1789762835.176091:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308419930500 time 308420030055 state 0x2 ref 0 00000100:00100000:1.0:1789762835.178343:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762835.178388:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067398016 00000100:00100000:0.0:1789762835.178396:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067398016 00000100:00100000:0.0:1789762835.178432:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308419930500 time 308420030055 state 0x0 ref 1 00000100:00100000:0.0:1789762835.178442:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308520030055 time 308420030055 state 0x1 ref 1 00000100:00100000:1.0:1789762835.273079:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308520030055 time 308520145128 state 0x0 ref 1 00000100:00100000:1.0:1789762835.273105:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067398016:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762835.273142:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175d15 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.273173:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762835.273177:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762835.273182:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175d15 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.273193:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175d15 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.276412:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175d15 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.276429:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175d15 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.276436:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175d15 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.276442:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762835.276446:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175d15 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.276453:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175d15 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.276469:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175d15 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.276490:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcac1903/0xdf5a7ac59f175d15 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762835.276561:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067398016:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3463us (98216us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762835.276570:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 397 00000100:00100000:1.0:1789762835.276591:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308520030055 time 308520145128 state 0x2 ref 0 00000100:00100000:1.0:1789762835.279145:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762835.279225:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067398144 00000100:00100000:0.0:1789762835.279236:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067398144 00000100:00100000:0.0:1789762835.279287:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308520030055 time 308520145128 state 0x0 ref 1 00000100:00100000:0.0:1789762835.279298:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308620145128 time 308520145128 state 0x1 ref 1 00000100:00100000:0.0:1789762835.373169:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308620145128 time 308620232691 state 0x0 ref 1 00000100:00100000:0.0:1789762835.373195:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067398144:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762835.373231:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d1c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.373244:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762835.373248:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762835.373252:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d1c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.373263:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d1c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.375986:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d1c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.375996:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d1c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.376002:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d1c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.376008:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762835.376011:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d1c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.376028:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d1c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.376043:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d1c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.376051:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d1c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762835.376111:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067398144:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2927us (96970us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762835.376119:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 398 00000100:00100000:0.0:1789762835.376127:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308620145128 time 308620232691 state 0x2 ref 0 00000100:00100000:0.0:1789762835.378335:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762835.378376:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067398272 00000100:00100000:1.0:1789762835.378384:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067398272 00000100:00100000:1.0:1789762835.378420:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308620145128 time 308620232691 state 0x0 ref 1 00000100:00100000:1.0:1789762835.378431:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308720232691 time 308620232691 state 0x1 ref 1 00000001:00080000:2.0:1789762835.467339:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967381 is committed 00000100:00100000:3.0:1789762835.473254:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308720232691 time 308720321691 state 0x0 ref 1 00000100:00100000:3.0:1789762835.473276:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067398272:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762835.473310:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d23 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.473337:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762835.473342:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762835.473348:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d23 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.473364:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d23 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.476297:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d23 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.476312:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d23 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.476319:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d23 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.476325:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762835.476329:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d23 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.476336:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d23 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.476352:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d23 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.476371:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d23 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762835.476426:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067398272:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3155us (98088us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762835.476433:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 399 00000100:00100000:3.0:1789762835.476441:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308720232691 time 308720321691 state 0x2 ref 0 00000100:00100000:0.0:1789762835.478637:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762835.478691:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067398400 00000100:00100000:1.0:1789762835.478710:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067398400 00000100:00100000:1.0:1789762835.478759:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308720232691 time 308720321691 state 0x0 ref 1 00000100:00100000:1.0:1789762835.478774:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308820321691 time 308720321691 state 0x1 ref 1 00000100:00100000:1.0:1789762835.573334:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308820321691 time 308820399585 state 0x0 ref 1 00000100:00100000:1.0:1789762835.573361:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067398400:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762835.573410:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175d2a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.573424:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762835.573428:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762835.573432:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175d2a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.573444:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175d2a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.576053:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175d2a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.576068:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175d2a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.576077:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175d2a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.576094:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1a95c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762835.576098:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175d2a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.576105:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175d2a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.576118:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175d2a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.576123:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd3f7def/0xdf5a7ac59f175d2a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762835.576179:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067398400:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2827us (97541us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762835.576189:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 400 00000100:00100000:0.0:1789762835.576197:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308820321691 time 308820399585 state 0x2 ref 0 00000100:00100000:0.0:1789762835.578389:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762835.578434:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067398528 00000100:00100000:1.0:1789762835.578442:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067398528 00000100:00100000:1.0:1789762835.578477:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308820321691 time 308820399585 state 0x0 ref 1 00000100:00100000:1.0:1789762835.578488:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308920399585 time 308820399585 state 0x1 ref 1 00000100:00100000:0.0:1789762835.673444:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308920399585 time 308920509030 state 0x0 ref 1 00000100:00100000:0.0:1789762835.673470:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067398528:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762835.673509:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d31 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.673545:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762835.673549:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762835.673554:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d31 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.673565:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d31 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.676563:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d31 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.676593:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d31 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.676604:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d31 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.676613:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762835.676627:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d31 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.676638:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d31 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.676657:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d31 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.676666:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175d31 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762835.676725:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067398528:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3263us (98335us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762835.676732:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 401 00000100:00100000:1.0:1789762835.676741:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308920399585 time 308920509030 state 0x2 ref 0 00000100:00100000:0.0:1789762835.678937:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762835.678979:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067398656 00000100:00100000:3.0:1789762835.678989:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067398656 00000100:00100000:3.0:1789762835.679027:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 308920399585 time 308920509030 state 0x0 ref 1 00000100:00100000:3.0:1789762835.679037:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309020509030 time 308920509030 state 0x1 ref 1 00000100:00100000:0.0:1789762835.773523:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309020509030 time 309020588668 state 0x0 ref 1 00000100:00100000:0.0:1789762835.773547:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067398656:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762835.773598:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175d38 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.773617:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762835.773623:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762835.773630:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175d38 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.773644:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175d38 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.776597:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175d38 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.776612:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175d38 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.776622:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175d38 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.776630:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762835.776646:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175d38 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.776657:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175d38 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.776675:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175d38 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.776684:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f175d38 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762835.776740:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067398656:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3200us (97802us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762835.776752:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 402 00000100:00100000:1.0:1789762835.776763:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309020509030 time 309020588668 state 0x2 ref 0 00000100:00100000:2.0:1789762835.778801:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762835.778843:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067398784 00000100:00100000:0.0:1789762835.778850:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067398784 00000100:00100000:0.0:1789762835.778889:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309020509030 time 309020588668 state 0x0 ref 1 00000100:00100000:0.0:1789762835.778899:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309120588668 time 309020588668 state 0x1 ref 1 00000100:00100000:0.0:1789762835.873631:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309120588668 time 309120695108 state 0x0 ref 1 00000100:00100000:0.0:1789762835.873658:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067398784:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762835.873696:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175d3f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.873711:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762835.873714:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762835.873719:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175d3f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.873730:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175d3f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.876521:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175d3f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.876532:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175d3f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.876540:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175d3f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.876559:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762835.876563:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175d3f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.876570:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175d3f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.876597:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175d3f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.876603:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f175d3f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762835.876651:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067398784:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3004us (97850us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762835.876658:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 403 00000100:00100000:0.0:1789762835.876666:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309120588668 time 309120695108 state 0x2 ref 0 00000100:00100000:2.0:1789762835.878891:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762835.878937:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067398912 00000100:00100000:1.0:1789762835.878945:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067398912 00000100:00100000:1.0:1789762835.878982:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309120588668 time 309120695108 state 0x0 ref 1 00000100:00100000:1.0:1789762835.878993:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309220695108 time 309120695108 state 0x1 ref 1 00000100:00100000:0.0:1789762835.968107:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff70df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074680448:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762835.968176:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762835.968225:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074680448 00000100:00100000:2.0:1789762835.968234:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074680448 00000100:00100000:2.0:1789762835.968284:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:2.0:1789762835.968293:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2528700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074680448:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762835.968333:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762835.968391:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2528700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074680448:12345-0@lo:41:osp-pre-1-0.0 Request processed in 99us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762835.968397:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:0.0:1789762835.968440:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff70df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074680448:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762835.973676:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309220695108 time 309220746361 state 0x0 ref 1 00000100:00100000:1.0:1789762835.973691:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067398912:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762835.973723:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d46 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.973737:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762835.973741:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762835.973745:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d46 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.973755:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d46 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.976730:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d46 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.976745:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d46 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.976755:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d46 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.976763:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762835.976768:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d46 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.976779:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d46 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.976809:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d46 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.976818:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d46 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762835.976865:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067398912:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3176us (97975us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762835.976873:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 404 00000100:00100000:1.0:1789762835.976880:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309220695108 time 309220746361 state 0x2 ref 0 00000100:00100000:2.0:1789762835.979022:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762835.979055:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067399040 00000100:00100000:1.0:1789762835.979061:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067399040 00000100:00100000:1.0:1789762835.979092:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309220695108 time 309220746361 state 0x0 ref 1 00000100:00100000:1.0:1789762835.979102:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309320746361 time 309220746361 state 0x1 ref 1 00000100:00100000:1.0:1789762836.073773:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309320746361 time 309320836560 state 0x0 ref 1 00000100:00100000:1.0:1789762836.073798:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067399040:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762836.073835:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d4d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.073848:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.073864:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.073869:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d4d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.073880:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d4d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.076858:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d4d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.076875:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d4d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.076887:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d4d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.076896:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762836.076901:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d4d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.076910:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d4d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.076945:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d4d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.076952:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d4d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.077011:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067399040:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3223us (97990us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762836.077019:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 405 00000100:00100000:1.0:1789762836.077028:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309320746361 time 309320836560 state 0x2 ref 0 00000100:00100000:0.0:1789762836.078977:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762836.079011:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067399168 00000100:00100000:1.0:1789762836.079019:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067399168 00000100:00100000:1.0:1789762836.079053:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309320746361 time 309320836560 state 0x0 ref 1 00000100:00100000:1.0:1789762836.079063:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309420836560 time 309320836560 state 0x1 ref 1 00010000:00010000:2.0:1789762836.096042:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762836.173847:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309420836560 time 309420912775 state 0x0 ref 1 00000100:00100000:3.0:1789762836.173878:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067399168:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762836.173939:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d54 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.173954:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762836.173958:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762836.173963:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d54 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.173991:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d54 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.176818:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d54 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.176829:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d54 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.176835:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d54 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.176840:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762836.176843:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d54 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.176850:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d54 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.176863:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d54 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.176868:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d54 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762836.176951:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067399168:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3084us (97974us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762836.176960:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 406 00000100:00100000:3.0:1789762836.176969:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309420836560 time 309420912775 state 0x2 ref 0 00000100:00100000:2.0:1789762836.178778:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762836.178823:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067399296 00000100:00100000:1.0:1789762836.178831:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067399296 00000100:00100000:1.0:1789762836.178867:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309420836560 time 309420912775 state 0x0 ref 1 00000100:00100000:1.0:1789762836.178878:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309520912775 time 309420912775 state 0x1 ref 1 00000100:00100000:0.0:1789762836.273994:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309520912775 time 309521043365 state 0x0 ref 1 00000100:00100000:0.0:1789762836.274042:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067399296:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762836.274082:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d5b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.274096:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762836.274100:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762836.274105:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d5b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.274116:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d5b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.277040:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d5b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.277051:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d5b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.277057:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d5b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.277063:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762836.277067:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d5b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.277073:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d5b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.277096:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d5b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.277102:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d5b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762836.277148:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067399296:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3115us (98369us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762836.277155:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 407 00000100:00100000:0.0:1789762836.277162:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309520912775 time 309521043365 state 0x2 ref 0 00000100:00100000:0.0:1789762836.279027:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762836.279093:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067399424 00000100:00100000:1.0:1789762836.279103:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067399424 00000100:00100000:1.0:1789762836.279148:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309520912775 time 309521043365 state 0x0 ref 1 00000100:00100000:1.0:1789762836.279158:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309621043365 time 309521043365 state 0x1 ref 1 00000100:00100000:0.0:1789762836.374060:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309621043365 time 309621124895 state 0x0 ref 1 00000100:00100000:0.0:1789762836.374087:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067399424:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762836.374131:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d62 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.374145:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762836.374149:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762836.374153:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d62 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.374164:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d62 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.377007:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d62 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.377024:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d62 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.377030:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d62 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.377036:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762836.377039:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d62 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.377046:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d62 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.377059:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d62 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.377064:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f175d62 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.377113:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067399424:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3037us (98086us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762836.377121:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 408 00000100:00100000:1.0:1789762836.377135:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309621043365 time 309621124895 state 0x2 ref 0 00000100:00100000:2.0:1789762836.378938:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762836.378979:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067399552 00000100:00100000:3.0:1789762836.378988:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067399552 00000100:00100000:3.0:1789762836.379023:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309621043365 time 309621124895 state 0x0 ref 1 00000100:00100000:3.0:1789762836.379036:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309721124895 time 309621124895 state 0x1 ref 1 00000100:00100000:0.0:1789762836.474130:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309721124895 time 309721195200 state 0x0 ref 1 00000100:00100000:0.0:1789762836.474152:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067399552:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762836.474189:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d69 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.474203:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762836.474206:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762836.474211:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d69 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.474222:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d69 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.476864:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d69 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.476887:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d69 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.476897:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d69 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.476906:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762836.476912:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d69 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.476950:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d69 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.476965:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d69 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.476971:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d69 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.477019:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067399552:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2874us (98082us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762836.477026:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 409 00000100:00100000:1.0:1789762836.477034:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309721124895 time 309721195200 state 0x2 ref 0 00000100:00100000:2.0:1789762836.478840:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762836.478890:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067399680 00000100:00100000:0.0:1789762836.478898:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067399680 00000100:00100000:0.0:1789762836.478965:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309721124895 time 309721195200 state 0x0 ref 1 00000100:00100000:0.0:1789762836.478978:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309821195200 time 309721195200 state 0x1 ref 1 00000100:00100000:0.0:1789762836.574221:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309821195200 time 309821279690 state 0x0 ref 1 00000100:00100000:0.0:1789762836.574246:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8eddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067399680:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762836.574299:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d70 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.574313:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762836.574316:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762836.574321:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d70 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.574331:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d70 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.577111:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d70 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.577125:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d70 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.577136:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d70 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.577146:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762836.577151:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d70 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.577162:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d70 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.577193:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d70 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.577203:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f175d70 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762836.577258:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8eddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067399680:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3021us (98417us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762836.577269:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 410 00000100:00100000:0.0:1789762836.577277:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309821195200 time 309821279690 state 0x2 ref 0 00000100:00100000:1.0:1789762836.579117:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762836.579159:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067399808 00000100:00100000:0.0:1789762836.579167:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067399808 00000100:00100000:0.0:1789762836.579201:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309821195200 time 309821279690 state 0x0 ref 1 00000100:00100000:0.0:1789762836.579211:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309921279690 time 309821279690 state 0x1 ref 1 00000100:00100000:1.0:1789762836.674326:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309921279690 time 309921391658 state 0x0 ref 1 00000100:00100000:1.0:1789762836.674354:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067399808:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762836.674390:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d77 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.674404:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.674407:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.674412:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d77 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.674424:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d77 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.677305:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d77 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.677316:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d77 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.677323:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d77 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.677329:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762836.677332:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d77 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.677350:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d77 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.677365:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d77 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.677370:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d77 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.677415:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067399808:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3073us (98299us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762836.677423:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 411 00000100:00100000:1.0:1789762836.677430:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309921279690 time 309921391658 state 0x2 ref 0 00000100:00100000:1.0:1789762836.679646:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762836.679697:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067399936 00000100:00100000:0.0:1789762836.679705:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067399936 00000100:00100000:0.0:1789762836.679743:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 309921279690 time 309921391658 state 0x0 ref 1 00000100:00100000:0.0:1789762836.679754:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310021391658 time 309921391658 state 0x1 ref 1 00000100:00100000:1.0:1789762836.774432:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310021391658 time 310021497557 state 0x0 ref 1 00000100:00100000:1.0:1789762836.774457:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067399936:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762836.774495:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d7e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.774509:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762836.774513:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.774517:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d7e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.774541:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d7e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.777123:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d7e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.777134:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d7e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.777141:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d7e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.777146:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762836.777150:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d7e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.777156:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d7e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.777171:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d7e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.777175:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d7e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.777233:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067399936:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2787us (97588us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762836.777241:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 412 00000100:00100000:1.0:1789762836.777249:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310021391658 time 310021497557 state 0x2 ref 0 00000100:00100000:1.0:1789762836.779060:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762836.779100:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067400064 00000100:00100000:0.0:1789762836.779109:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067400064 00000100:00100000:0.0:1789762836.779164:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310021391658 time 310021497557 state 0x0 ref 1 00000100:00100000:0.0:1789762836.779179:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310121497557 time 310021497557 state 0x1 ref 1 00000100:00100000:3.0:1789762836.874532:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310121497557 time 310121596713 state 0x0 ref 1 00000100:00100000:3.0:1789762836.874559:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067400064:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762836.874609:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d85 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.874624:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762836.874627:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762836.874632:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d85 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.874642:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d85 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.877724:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d85 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.877734:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d85 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.877742:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d85 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.877748:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762836.877752:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d85 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.877758:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d85 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.877772:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d85 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.877778:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d0f2c75d/0xdf5a7ac59f175d85 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762836.877826:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067400064:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3278us (98764us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762836.877832:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 413 00000100:00100000:3.0:1789762836.877847:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310121497557 time 310121596713 state 0x2 ref 0 00000100:00100000:2.0:1789762836.879703:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762836.879749:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067400192 00000100:00100000:1.0:1789762836.879756:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067400192 00000100:00100000:1.0:1789762836.879790:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310121497557 time 310121596713 state 0x0 ref 1 00000100:00100000:1.0:1789762836.879801:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310221596713 time 310121596713 state 0x1 ref 1 00000100:00100000:1.0:1789762836.974604:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310221596713 time 310221666545 state 0x0 ref 1 00000100:00100000:1.0:1789762836.974630:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8ede300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067400192:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762836.974665:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d8c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.974678:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.974682:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.974686:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d8c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.974697:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d8c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.977283:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d8c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.977294:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d8c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.977301:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d8c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.977307:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762836.977311:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d8c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.977317:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d8c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.977339:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d8c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.977345:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d8c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.977394:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8ede300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067400192:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2772us (97691us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762836.977402:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 414 00000100:00100000:1.0:1789762836.977417:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310221596713 time 310221666545 state 0x2 ref 0 00000100:00100000:2.0:1789762836.979472:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762836.979514:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067400320 00000100:00100000:1.0:1789762836.979522:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067400320 00000100:00100000:1.0:1789762836.979556:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310221596713 time 310221666545 state 0x0 ref 1 00000100:00100000:1.0:1789762836.979580:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310321666545 time 310221666545 state 0x1 ref 1 00000100:00100000:1.0:1789762836.992057:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d252a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074680576:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762836.992099:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762836.992140:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074680576 00000100:00100000:2.0:1789762836.992146:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074680576 00000100:00100000:2.0:1789762836.992180:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:2.0:1789762836.992190:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074680576:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762836.992220:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762836.992265:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074680576:12345-0@lo:41:osp-pre-0-1.0 Request processed in 77us (165us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762836.992271:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00100000:1.0:1789762836.992306:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d252a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074680576:0@lo:41:osp-pre-0-1.0 00000001:00080000:0.0:1789762837.001068:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967792 is committed 00000100:00100000:1.0:1789762837.074692:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310321666545 time 310321758595 state 0x0 ref 1 00000100:00100000:1.0:1789762837.074732:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8ede680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067400320:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762837.074768:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d93 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789762837.074776:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762837.074790:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762837.074794:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762837.074799:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d93 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.074810:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d93 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.077938:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d93 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.077950:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d93 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.077957:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d93 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.077964:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762837.077967:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d93 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.077974:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d93 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.077987:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d93 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.077992:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175d93 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.078038:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8ede680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067400320:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3329us (98566us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762837.078044:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 415 00000100:00100000:1.0:1789762837.078058:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310321666545 time 310321758595 state 0x2 ref 0 00000100:00100000:2.0:1789762837.080237:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762837.080278:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067400448 00000100:00100000:3.0:1789762837.080286:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067400448 00000100:00100000:3.0:1789762837.080322:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310321666545 time 310321758595 state 0x0 ref 1 00000100:00100000:3.0:1789762837.080332:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310421758595 time 310321758595 state 0x1 ref 1 00010000:00010000:1.0:1789762837.120021:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762837.174766:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310421758595 time 310421833793 state 0x0 ref 1 00000100:00100000:0.0:1789762837.174790:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067400448:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762837.174826:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175d9a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.174840:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.174845:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.174850:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175d9a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.174862:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175d9a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.182899:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175d9a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.182912:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175d9a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.182936:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175d9a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.182949:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762837.182954:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175d9a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.182962:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175d9a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.182980:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175d9a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.182985:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175d9a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762837.183056:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067400448:12345-192.168.203.37@tcp:3:dd.0 Request processed in 8270us (102814us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762837.183068:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 416 00000100:00100000:0.0:1789762837.183078:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310421758595 time 310421833793 state 0x2 ref 0 00000100:00100000:0.0:1789762837.185030:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762837.185066:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067400576 00000100:00100000:0.0:1789762837.185074:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067400576 00000100:00100000:0.0:1789762837.185115:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310421758595 time 310421833793 state 0x0 ref 1 00000100:00100000:0.0:1789762837.185127:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310521833793 time 310421833793 state 0x1 ref 1 00000100:00100000:0.0:1789762837.274836:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310521833793 time 310521900467 state 0x0 ref 1 00000100:00100000:0.0:1789762837.274863:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067400576:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762837.274902:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175da1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.274941:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.274945:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.274951:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175da1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.274962:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175da1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.277991:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175da1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.278002:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175da1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.278009:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175da1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.278014:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762837.278018:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175da1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.278025:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175da1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.278041:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175da1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.278049:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f175da1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.278115:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067400576:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3261us (93087us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762837.278137:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 417 00000100:00100000:1.0:1789762837.278145:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310521833793 time 310521900467 state 0x2 ref 0 00000100:00100000:3.0:1789762837.280016:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762837.280049:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067400704 00000100:00100000:1.0:1789762837.280059:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067400704 00000100:00100000:1.0:1789762837.280100:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310521833793 time 310521900467 state 0x0 ref 1 00000100:00100000:1.0:1789762837.280111:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310621900467 time 310521900467 state 0x1 ref 1 00000100:00100000:1.0:1789762837.374971:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310621900467 time 310622033047 state 0x0 ref 1 00000100:00100000:1.0:1789762837.375000:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067400704:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762837.375050:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175da8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.375067:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762837.375071:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762837.375077:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175da8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.375092:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175da8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.377860:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175da8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.377884:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175da8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.377891:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175da8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.377897:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762837.377901:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175da8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.377908:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175da8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.377938:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175da8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.377944:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175da8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.377990:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067400704:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3000us (97974us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762837.377996:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 418 00000100:00100000:1.0:1789762837.378003:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310621900467 time 310622033047 state 0x2 ref 0 00000100:00100000:3.0:1789762837.379813:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762837.379863:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067400832 00000100:00100000:0.0:1789762837.379873:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067400832 00000100:00100000:0.0:1789762837.379946:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310621900467 time 310622033047 state 0x0 ref 1 00000100:00100000:0.0:1789762837.379963:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310722033047 time 310622033047 state 0x1 ref 1 00000100:00100000:0.0:1789762837.475024:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310722033047 time 310722087485 state 0x0 ref 1 00000100:00100000:0.0:1789762837.475048:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067400832:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762837.475084:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f175daf lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.475111:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.475114:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.475119:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f175daf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.475130:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f175daf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.477726:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f175daf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.477743:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f175daf lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.477751:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f175daf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.477757:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762837.477761:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f175daf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.477767:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f175daf lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.477782:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f175daf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.477788:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f175daf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.477838:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067400832:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2799us (98026us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762837.477845:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 419 00000100:00100000:1.0:1789762837.477853:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310722033047 time 310722087485 state 0x2 ref 0 00000100:00100000:3.0:1789762837.479689:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762837.479725:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067400960 00000100:00100000:1.0:1789762837.479740:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067400960 00000100:00100000:1.0:1789762837.479774:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310722033047 time 310722087485 state 0x0 ref 1 00000100:00100000:1.0:1789762837.479785:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310822087485 time 310722087485 state 0x1 ref 1 00000001:00080000:0.0:1789762837.514384:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967397 is committed 00000100:00100000:3.0:1789762837.575122:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310822087485 time 310822185227 state 0x0 ref 1 00000100:00100000:3.0:1789762837.575146:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067400960:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762837.575191:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175db6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762837.575205:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762837.575208:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762837.575213:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175db6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762837.575224:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175db6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.577988:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175db6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.578000:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175db6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.578008:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175db6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.578014:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762837.578017:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175db6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.578024:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175db6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.578039:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175db6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.578054:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000035d21925/0xdf5a7ac59f175db6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762837.578110:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067400960:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2972us (98420us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762837.578118:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 420 00000100:00100000:0.0:1789762837.578126:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310822087485 time 310822185227 state 0x2 ref 0 00000100:00100000:2.0:1789762837.580057:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762837.580106:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067401088 00000100:00100000:1.0:1789762837.580114:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067401088 00000100:00100000:1.0:1789762837.580150:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310822087485 time 310822185227 state 0x0 ref 1 00000100:00100000:1.0:1789762837.580161:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310922185227 time 310822185227 state 0x1 ref 1 00000100:00100000:0.0:1789762837.675247:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310922185227 time 310922312620 state 0x0 ref 1 00000100:00100000:0.0:1789762837.675273:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067401088:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762837.675310:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dbd lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.675325:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.675329:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.675336:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dbd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.675351:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dbd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.677894:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dbd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.677948:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dbd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.677959:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dbd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.677967:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762837.677972:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dbd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.677982:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dbd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.678000:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dbd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.678009:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dbd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762837.678064:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067401088:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2800us (98006us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762837.678073:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 421 00000100:00100000:0.0:1789762837.678084:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310922185227 time 310922312620 state 0x2 ref 0 00000100:00100000:3.0:1789762837.679944:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762837.679987:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067401216 00000100:00100000:1.0:1789762837.679996:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067401216 00000100:00100000:1.0:1789762837.680039:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 310922185227 time 310922312620 state 0x0 ref 1 00000100:00100000:1.0:1789762837.680054:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311022312620 time 310922312620 state 0x1 ref 1 00000100:00100000:0.0:1789762837.775343:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311022312620 time 311022404130 state 0x0 ref 1 00000100:00100000:0.0:1789762837.775385:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067401216:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762837.775422:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dc4 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.775437:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762837.775440:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.775445:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dc4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.775458:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dc4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.777889:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dc4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.777899:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dc4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.777905:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dc4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.777911:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762837.777914:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dc4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.777954:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dc4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.777981:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dc4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.777987:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dc4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762837.778032:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067401216:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2655us (98090us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762837.778040:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 422 00000100:00100000:0.0:1789762837.778049:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311022312620 time 311022404130 state 0x2 ref 0 00000100:00100000:3.0:1789762837.780245:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762837.780283:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067401344 00000100:00100000:0.0:1789762837.780291:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067401344 00000100:00100000:0.0:1789762837.780327:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311022312620 time 311022404130 state 0x0 ref 1 00000100:00100000:0.0:1789762837.780338:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311122404130 time 311022404130 state 0x1 ref 1 00000100:00100000:0.0:1789762837.875402:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311122404130 time 311122467957 state 0x0 ref 1 00000100:00100000:0.0:1789762837.875427:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067401344:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762837.875462:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dcb lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.875475:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.875491:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.875496:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dcb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.875508:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dcb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.878298:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dcb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.878308:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dcb lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.878315:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dcb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.878320:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762837.878323:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dcb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.878329:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dcb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.878342:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dcb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.878347:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dcb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762837.878401:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067401344:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2983us (98158us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762837.878408:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 423 00000100:00100000:0.0:1789762837.878416:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311122404130 time 311122467957 state 0x2 ref 0 00000100:00100000:3.0:1789762837.880501:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762837.880542:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067401472 00000100:00100000:0.0:1789762837.880551:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067401472 00000100:00100000:0.0:1789762837.880601:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311122404130 time 311122467957 state 0x0 ref 1 00000100:00100000:0.0:1789762837.880611:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311222467957 time 311122467957 state 0x1 ref 1 00000100:00100000:2.0:1789762837.959911:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762837.959991:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762837.960006:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067401728 00000100:00100000:2.0:1789762837.960014:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762837.960016:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067401728 00000100:00100000:2.0:1789762837.960041:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762837.960057:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067401600 00000100:00100000:3.0:1789762837.960057:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067401856 00000100:00100000:1.0:1789762837.960065:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067401600 00000100:00100000:3.0:1789762837.960065:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067401856 00000100:00100000:2.0:1789762837.960071:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762837.960081:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067402112 00000100:00100000:0.0:1789762837.960086:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067402112 00000100:00100000:2.0:1789762837.960107:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762837.960123:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067402240 00000100:00100000:1.0:1789762837.960130:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 347 00000100:00100000:0.0:1789762837.960131:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 428 00000100:00100000:3.0:1789762837.960131:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067402240 00000100:00100000:2.0:1789762837.960144:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 429 00000100:00100000:1.0:1789762837.960150:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1578c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067401600:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762837.960152:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2396:x1876702067401728:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762837.960161:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067401856:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762837.960166:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 430 00000100:00100000:3.0:1789762837.960178:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067402112:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762837.960218:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1578c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067401600:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 74us (224us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762837.960219:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2396:x1876702067401728:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 73us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762837.960222:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067401856:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 64us (205us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762837.960235:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067402112:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 55us (160us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762837.960241:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 428 00000100:00100000:1.0:1789762837.960242:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 347 00000100:00100000:3.0:1789762837.960256:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 430 00000100:00100000:2.0:1789762837.960265:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 429 00000100:00100000:0.0:1789762837.960276:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 431 00000100:00100000:0.0:1789762837.960285:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067402240:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:3.0:1789762837.960290:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762837.960297:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762837.960313:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067402240:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 29us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762837.960316:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067401984 00000100:00100000:3.0:1789762837.960321:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067401984 00000100:00100000:0.0:1789762837.960334:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 431 00000100:00080000:1.0:1789762837.960365:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762837.960369:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 167 00000100:00100000:3.0:1789762837.960379:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067401984:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762837.960405:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067401984:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 30us (366us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762837.960410:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 167 00000100:00100000:0.0:1789762837.975448:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311222467957 time 311222518783 state 0x0 ref 1 00000100:00100000:0.0:1789762837.975464:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067401472:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762837.975491:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dd2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.975505:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.975508:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762837.975513:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dd2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.975523:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dd2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.978265:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dd2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.978292:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dd2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.978304:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dd2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.978313:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762837.978318:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dd2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.978329:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dd2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.978349:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dd2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.978357:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175dd2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.978409:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067401472:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2946us (97907us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762837.978419:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 424 00000100:00100000:1.0:1789762837.978430:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311222467957 time 311222518783 state 0x2 ref 0 00000100:00100000:2.0:1789762837.980259:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762837.980310:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067402368 00000100:00100000:0.0:1789762837.980317:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067402368 00000100:00100000:0.0:1789762837.980354:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311222467957 time 311222518783 state 0x0 ref 1 00000100:00100000:0.0:1789762837.980367:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311322518783 time 311222518783 state 0x1 ref 1 00000100:00100000:1.0:1789762838.075585:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311322518783 time 311322635495 state 0x0 ref 1 00000100:00100000:1.0:1789762838.075628:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8eddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067402368:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762838.075671:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dd9 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.075687:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.075692:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.075699:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dd9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.075713:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dd9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.078845:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dd9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.078858:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dd9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.078868:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dd9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.078876:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762838.078881:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dd9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.078891:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dd9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.078933:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dd9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.078944:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dd9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762838.079001:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8eddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067402368:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3382us (98741us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762838.079011:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 425 00000100:00100000:1.0:1789762838.079022:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311322518783 time 311322635495 state 0x2 ref 0 00000100:00100000:3.0:1789762838.080770:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762838.080813:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067402496 00000100:00100000:0.0:1789762838.080822:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067402496 00000100:00100000:0.0:1789762838.080861:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311322518783 time 311322635495 state 0x0 ref 1 00000100:00100000:0.0:1789762838.080875:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311422635495 time 311322635495 state 0x1 ref 1 00010000:00010000:3.0:1789762838.144028:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762838.175662:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311422635495 time 311422723257 state 0x0 ref 1 00000100:00100000:1.0:1789762838.175692:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067402496:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762838.175731:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175de0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.175744:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.175748:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.175752:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175de0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.175763:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175de0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.178614:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175de0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.178636:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175de0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.178647:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175de0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.178657:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762838.178663:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175de0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.178674:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175de0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.178694:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175de0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.178700:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175de0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762838.178751:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067402496:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3070us (97981us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762838.178759:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 426 00000100:00100000:1.0:1789762838.178768:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311422635495 time 311422723257 state 0x2 ref 0 00000100:00100000:3.0:1789762838.180817:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762838.180859:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067402624 00000100:00100000:1.0:1789762838.180868:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067402624 00000100:00100000:1.0:1789762838.180909:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311422635495 time 311422723257 state 0x0 ref 1 00000100:00100000:1.0:1789762838.180937:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311522723257 time 311422723257 state 0x1 ref 1 00000100:00100000:3.0:1789762838.275728:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311522723257 time 311522795097 state 0x0 ref 1 00000100:00100000:3.0:1789762838.275753:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067402624:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762838.275803:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175de7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.275816:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762838.275820:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762838.275824:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175de7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.275835:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175de7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.278504:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175de7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.278513:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175de7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.278519:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175de7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.278524:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5870600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762838.278527:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175de7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.278534:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175de7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.278571:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175de7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.278579:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7f689ce/0xdf5a7ac59f175de7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762838.278625:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067402624:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2882us (97810us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762838.278632:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 427 00000100:00100000:3.0:1789762838.278639:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311522723257 time 311522795097 state 0x2 ref 0 00000100:00100000:3.0:1789762838.280594:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762838.280636:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067402752 00000100:00100000:1.0:1789762838.280644:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067402752 00000100:00100000:1.0:1789762838.280679:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311522723257 time 311522795097 state 0x0 ref 1 00000100:00100000:1.0:1789762838.280691:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311622795097 time 311522795097 state 0x1 ref 1 00000100:00100000:1.0:1789762838.375787:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311622795097 time 311622852035 state 0x0 ref 1 00000100:00100000:1.0:1789762838.375810:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067402752:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762838.375845:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dee lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.375859:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.375863:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.375868:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dee lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.375879:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dee lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.378790:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dee lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.378801:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dee lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.378809:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dee lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.378815:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762838.378818:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dee lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.378825:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dee lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.378839:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dee lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.378844:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f175dee lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762838.378902:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067402752:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3098us (98307us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762838.378957:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 428 00000100:00100000:2.0:1789762838.378966:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311622795097 time 311622852035 state 0x2 ref 0 00000100:00100000:0.0:1789762838.380848:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762838.380891:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067402880 00000100:00100000:1.0:1789762838.380898:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067402880 00000100:00100000:1.0:1789762838.380952:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311622795097 time 311622852035 state 0x0 ref 1 00000100:00100000:1.0:1789762838.380967:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311722852035 time 311622852035 state 0x1 ref 1 00000100:00100000:0.0:1789762838.475900:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311722852035 time 311722964629 state 0x0 ref 1 00000100:00100000:0.0:1789762838.475972:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2528a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067402880:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762838.476010:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001990e7bb/0xdf5a7ac59f175df5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.476025:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762838.476028:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762838.476033:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001990e7bb/0xdf5a7ac59f175df5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.476044:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001990e7bb/0xdf5a7ac59f175df5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.478629:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001990e7bb/0xdf5a7ac59f175df5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.478645:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001990e7bb/0xdf5a7ac59f175df5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.478653:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001990e7bb/0xdf5a7ac59f175df5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.478660:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ffb8e000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762838.478663:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001990e7bb/0xdf5a7ac59f175df5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.478671:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001990e7bb/0xdf5a7ac59f175df5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.478684:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001990e7bb/0xdf5a7ac59f175df5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.478694:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001990e7bb/0xdf5a7ac59f175df5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762838.478740:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2528a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067402880:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2826us (97894us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762838.478747:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 429 00000100:00100000:1.0:1789762838.478755:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311722852035 time 311722964629 state 0x2 ref 0 00000100:00100000:1.0:1789762838.480573:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762838.480606:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067403008 00000100:00100000:3.0:1789762838.480614:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067403008 00000100:00100000:3.0:1789762838.480649:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311722852035 time 311722964629 state 0x0 ref 1 00000100:00100000:3.0:1789762838.480660:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311822964629 time 311722964629 state 0x1 ref 1 00000100:00100000:0.0:1789762838.575998:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311822964629 time 311823063764 state 0x0 ref 1 00000100:00100000:0.0:1789762838.576021:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067403008:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762838.576063:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175dfc lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.576077:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762838.576081:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762838.576085:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175dfc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.576110:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175dfc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.578899:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175dfc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.578908:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175dfc lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.578932:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175dfc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.578938:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ffb8fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762838.578941:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175dfc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.578948:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175dfc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.578961:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175dfc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.578966:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175dfc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762838.579022:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067403008:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3009us (98464us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762838.579029:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 430 00000100:00100000:0.0:1789762838.579037:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311822964629 time 311823063764 state 0x2 ref 0 00000100:00100000:3.0:1789762838.580813:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762838.580856:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067403136 00000100:00100000:2.0:1789762838.580864:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067403136 00000100:00100000:2.0:1789762838.580900:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311822964629 time 311823063764 state 0x0 ref 1 00000100:00100000:2.0:1789762838.580911:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311923063764 time 311823063764 state 0x1 ref 1 00000100:00100000:0.0:1789762838.676103:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311923063764 time 311923167409 state 0x0 ref 1 00000100:00100000:0.0:1789762838.676136:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067403136:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762838.676171:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e03 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.676185:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762838.676189:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762838.676194:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e03 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.676205:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e03 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.679181:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e03 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.679192:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e03 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.679219:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e03 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.679227:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ffb8fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762838.679233:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e03 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.679244:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e03 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.679264:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e03 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.679271:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e03 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762838.679319:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067403136:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3201us (98507us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762838.679325:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 431 00000100:00100000:0.0:1789762838.679333:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311923063764 time 311923167409 state 0x2 ref 0 00000100:00100000:2.0:1789762838.681164:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762838.681204:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067403264 00000100:00100000:1.0:1789762838.681212:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067403264 00000100:00100000:1.0:1789762838.681251:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 311923063764 time 311923167409 state 0x0 ref 1 00000100:00100000:1.0:1789762838.681267:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312023167409 time 311923167409 state 0x1 ref 1 00000100:00100000:1.0:1789762838.776178:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312023167409 time 312023243457 state 0x0 ref 1 00000100:00100000:1.0:1789762838.776200:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067403264:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762838.776234:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e0a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.776250:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762838.776254:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.776261:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e0a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.776280:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e0a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.779251:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e0a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.779263:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e0a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.779270:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e0a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.779276:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762838.779279:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e0a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.779287:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e0a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.779301:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e0a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.779306:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e0a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762838.779357:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067403264:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3164us (98193us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762838.779365:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 432 00000100:00100000:1.0:1789762838.779374:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312023167409 time 312023243457 state 0x2 ref 0 00000100:00100000:1.0:1789762838.781818:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762838.781877:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067403392 00000100:00100000:0.0:1789762838.781888:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067403392 00000100:00100000:0.0:1789762838.781968:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312023167409 time 312023243457 state 0x0 ref 1 00000100:00100000:0.0:1789762838.781998:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312123243457 time 312023243457 state 0x1 ref 1 00000100:00100000:1.0:1789762838.876263:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312123243457 time 312123329260 state 0x0 ref 1 00000100:00100000:1.0:1789762838.876287:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067403392:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762838.876329:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e11 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.876342:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.876346:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.876351:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e11 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.876362:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e11 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.879144:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e11 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.879156:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e11 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.879162:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e11 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.879168:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d9230c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762838.879171:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e11 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.879178:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e11 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.879192:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e11 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.879197:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba62e277/0xdf5a7ac59f175e11 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762838.879242:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067403392:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2964us (97430us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762838.879265:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 433 00000100:00100000:0.0:1789762838.879273:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312123243457 time 312123329260 state 0x2 ref 0 00000100:00100000:3.0:1789762838.881079:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762838.881109:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067403520 00000100:00100000:0.0:1789762838.881116:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067403520 00000100:00100000:0.0:1789762838.881148:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312123243457 time 312123329260 state 0x0 ref 1 00000100:00100000:0.0:1789762838.881157:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312223329260 time 312123329260 state 0x1 ref 1 00000100:00100000:3.0:1789762838.976340:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312223329260 time 312223407169 state 0x0 ref 1 00000100:00100000:3.0:1789762838.976362:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067403520:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762838.976397:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f175e18 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.976421:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762838.976424:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762838.976429:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f175e18 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.976440:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f175e18 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.979273:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f175e18 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.979284:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f175e18 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.979291:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f175e18 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.979296:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998fc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762838.979300:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f175e18 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.979306:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f175e18 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.979321:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f175e18 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.979340:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f175e18 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762838.979395:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067403520:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3038us (98315us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762838.979404:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 434 00000100:00100000:0.0:1789762838.979415:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312223329260 time 312223407169 state 0x2 ref 0 00000100:00100000:2.0:1789762838.981200:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762838.981241:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067403648 00000100:00100000:1.0:1789762838.981249:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067403648 00000100:00100000:1.0:1789762838.981281:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312223329260 time 312223407169 state 0x0 ref 1 00000100:00100000:1.0:1789762838.981294:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312323407169 time 312223407169 state 0x1 ref 1 00000100:00100000:3.0:1789762839.040060:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1789762839.040077:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2529c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074680960:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762839.040081:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2528380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074681216:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762839.040087:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c8edd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074681088:0@lo:400:kworker.0 00000100:00100000:3.0:1789762839.040149:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762839.040153:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762839.040157:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074681344:0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.040190:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762839.040198:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762839.040211:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074680832:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762839.040215:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff615180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074681600:0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.040229:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff614380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074681856:0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.040234:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074681472:0@lo:400:kworker.0 00000100:00100000:3.0:1789762839.040256:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762839.040262:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762839.040282:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074681216 00000100:00100000:1.0:1789762839.040288:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074681216 00000100:00100000:3.0:1789762839.040290:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c8edea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074680704:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762839.040294:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074682112:0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.040294:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762839.040331:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff614e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074681984:0@lo:400:kworker.0 00000100:00100000:3.0:1789762839.040332:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762839.040338:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762839.040344:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074680960 00000100:00100000:1.0:1789762839.040348:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074680960 00000100:00100000:1.0:1789762839.040370:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074680832 00000100:00100000:2.0:1789762839.040370:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762839.040371:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff617100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074681728:0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.040373:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff614700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074682624:0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.040375:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074680832 00000100:00100000:1.0:1789762839.040398:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074680704 00000100:00100000:1.0:1789762839.040401:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074680704 00000100:00100000:2.0:1789762839.040405:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff614000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074682496:0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.040421:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:0.0:1789762839.040426:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762839.040428:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762839.040435:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074681216:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762839.040457:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074681344 00000100:00100000:2.0:1789762839.040464:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074681344 00002000:00000024:1.0:1789762839.040467:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439418880 pending 0 free 3754950656 avail 3752853504 00000100:00100000:3.0:1789762839.040473:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074682240:0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.040486:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074681088 00000100:00100000:0.0:1789762839.040491:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074681088 00000100:00100000:2.0:1789762839.040505:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074681600 00000100:00100000:2.0:1789762839.040511:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074681600 00000100:00100000:1.0:1789762839.040517:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074681216:12345-0@lo:13:osp-pre-0-1.0 Request processed in 82us (367us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762839.040521:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762839.040529:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 348 00000100:00100000:1.0:1789762839.040530:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00100000:0.0:1789762839.040544:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d157a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074681088:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.040544:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074682112 00000100:00100000:2.0:1789762839.040550:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074682112 00000100:00100000:3.0:1789762839.040568:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff617800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074682752:0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.040571:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:1.0:1789762839.040585:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074680960:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762839.040596:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074681984 00000100:00100000:2.0:1789762839.040601:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074681984 00002000:00000024:1.0:1789762839.040605:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 13631488 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762839.040611:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d157a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074681088:12345-0@lo:400:kworker.0 Request processed in 68us (456us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762839.040614:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762839.040618:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074682240 00000100:00100000:0.0:1789762839.040626:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 348 00000100:00100000:2.0:1789762839.040647:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074682240 00000100:00100000:1.0:1789762839.040651:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074680960:12345-0@lo:13:osp-pre-1-0.0 Request processed in 68us (462us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762839.040660:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00100000:3.0:1789762839.040662:0:3424:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294967792 gen 2 00000004:00080000:3.0:1789762839.040668:0:3424:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37fd9c1180, transno 4294967792 00000100:00100000:1.0:1789762839.040682:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:2.0:1789762839.040689:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:0.0:1789762839.040690:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:1.0:1789762839.040695:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074680832:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762839.040696:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2528380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074681216:0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789762839.040705:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 13631488 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762839.040709:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074680704:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762839.040710:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074681344:12345-0@lo:400:kworker.0 00002000:00000024:0.0:1789762839.040727:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439418880 pending 0 free 3754950656 avail 3752853504 00000100:00100000:1.0:1789762839.040733:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074680832:12345-0@lo:13:osp-pre-1-1.0 Request processed in 43us (476us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762839.040739:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:3.0:1789762839.040748:0:3424:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294967381 gen 2 00000004:00080000:3.0:1789762839.040754:0:3424:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37f6088380, transno 4294967381 00000100:00100000:1.0:1789762839.040766:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:3.0:1789762839.040777:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074680832:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762839.040778:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074681600:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.040778:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074681344:12345-0@lo:400:kworker.0 Request processed in 74us (578us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762839.040783:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074680704:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (451us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762839.040798:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:2.0:1789762839.040800:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00100000:3.0:1789762839.040808:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c8edea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074680704:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762839.040822:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074681600:12345-0@lo:400:kworker.0 Request processed in 49us (563us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762839.040829:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:0.0:1789762839.040832:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074681472 00000100:00100000:1.0:1789762839.040836:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00100000:0.0:1789762839.040837:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074681472 00000100:00100000:2.0:1789762839.040842:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074682112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762839.040848:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074681728 00000100:00100000:3.0:1789762839.040852:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074681728 00000100:00100000:1.0:1789762839.040858:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 435 00000100:00100000:1.0:1789762839.040868:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff617b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074681984:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.040875:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:0.0:1789762839.040886:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074681472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.040892:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074682112:12345-0@lo:400:kworker.0 Request processed in 52us (551us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762839.040892:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 289 00000100:00100000:2.0:1789762839.040900:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:3.0:1789762839.040904:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074681728:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.040943:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff617b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074681984:12345-0@lo:400:kworker.0 Request processed in 53us (552us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762839.040948:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:0.0:1789762839.040954:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074681472:12345-0@lo:400:kworker.0 Request processed in 70us (663us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762839.040958:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 435 00000100:00100000:2.0:1789762839.040964:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074682240:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.040969:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00100000:3.0:1789762839.040984:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074681728:12345-0@lo:400:kworker.0 Request processed in 81us (560us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762839.040991:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762839.040996:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 289 00000100:00080000:1.0:1789762839.040997:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762839.041003:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789762839.041006:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762839.041010:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074682240:12345-0@lo:400:kworker.0 Request processed in 49us (492us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762839.041012:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074682624 00000100:00100000:0.0:1789762839.041019:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074682624 00000100:00100000:2.0:1789762839.041025:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00100000:1.0:1789762839.041026:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762839.041029:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074682752 00000100:00100000:3.0:1789762839.041035:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074682752 00000100:00100000:2.0:1789762839.041048:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074681856 00000100:00100000:2.0:1789762839.041052:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074681856 00000100:00100000:1.0:1789762839.041054:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff615880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074682368:0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.041063:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:3.0:1789762839.041072:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:0.0:1789762839.041077:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074682624:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.041085:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762839.041090:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:3.0:1789762839.041090:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074682752:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.041100:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074681856:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.041124:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074681344:0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.041129:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074682624:12345-0@lo:400:kworker.0 Request processed in 55us (706us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762839.041140:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00100000:2.0:1789762839.041141:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074681856:12345-0@lo:400:kworker.0 Request processed in 41us (115us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762839.041141:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074682752:12345-0@lo:400:kworker.0 Request processed in 55us (528us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762839.041147:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff614380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074681856:0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.041155:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074682368 00000100:00100000:2.0:1789762839.041155:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00100000:3.0:1789762839.041156:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000100:00100000:0.0:1789762839.041160:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074682368 00000100:00080000:1.0:1789762839.041180:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762839.041184:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:2.0:1789762839.041189:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762839.041195:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074682368:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.041203:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074682496 00000100:00100000:0.0:1789762839.041207:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074682496 00000100:00100000:1.0:1789762839.041231:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 171 00000100:00100000:2.0:1789762839.041237:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2529c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074680960:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762839.041241:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074682368:12345-0@lo:400:kworker.0 Request processed in 48us (157us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762839.041244:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074682496:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762839.041257:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00100000:0.0:1789762839.041271:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff615180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074681600:0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.041273:0:3426:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967397 gen 2 00000004:00080000:2.0:1789762839.041286:0:3426:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967396 committed transno 4294967397 00000100:00100000:0.0:1789762839.041292:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074682112:0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.041293:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074682496:12345-0@lo:400:kworker.0 Request processed in 52us (101us total) trans 0 rc 0/0 00000004:00080000:2.0:1789762839.041302:0:3426:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967397 committed transno 4294967397 00000100:00100000:1.0:1789762839.041306:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 171 00000100:00100000:0.0:1789762839.041309:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff614700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074682624:0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.041323:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074681472:0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.041332:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c8edd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074681088:0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.041350:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff614e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074681984:0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.041351:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff615880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074682368:0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.041378:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff614000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074682496:0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.041400:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff617100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074681728:0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.041412:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074682240:0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.041419:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff617800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074682752:0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.076435:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312323407169 time 312323499049 state 0x0 ref 1 00000100:00100000:2.0:1789762839.076473:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067403648:12345-192.168.203.37@tcp:3:dd.0 00000100:00080000:1.0:1789762839.076492:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789762839.076510:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f175e1f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762839.076525:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762839.076528:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762839.076533:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f175e1f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762839.076544:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f175e1f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762839.079863:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f175e1f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762839.079878:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f175e1f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762839.079901:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f175e1f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762839.079906:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff934e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762839.079910:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f175e1f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762839.079934:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f175e1f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762839.079949:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f175e1f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762839.079955:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008102dc92/0xdf5a7ac59f175e1f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762839.080001:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067403648:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3550us (98801us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762839.080008:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 435 00000100:00100000:2.0:1789762839.080016:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312323407169 time 312323499049 state 0x2 ref 0 00000100:00100000:2.0:1789762839.081976:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762839.082027:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067403776 00000100:00100000:0.0:1789762839.082036:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067403776 00000100:00100000:0.0:1789762839.082072:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312323407169 time 312323499049 state 0x0 ref 1 00000100:00100000:0.0:1789762839.082082:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312423499049 time 312323499049 state 0x1 ref 1 00010000:00010000:2.0:1789762839.168010:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762839.176515:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312423499049 time 312423584244 state 0x0 ref 1 00000100:00100000:1.0:1789762839.176538:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067403776:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762839.176588:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e26 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.176605:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762839.176609:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762839.176616:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e26 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.176637:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e26 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.179996:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e26 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.180007:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e26 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.180014:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e26 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.180020:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762839.180024:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e26 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.180030:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e26 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.180043:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e26 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.180048:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e26 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762839.180099:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067403776:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3569us (98122us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762839.180106:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 436 00000100:00100000:1.0:1789762839.180114:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312423499049 time 312423584244 state 0x2 ref 0 00000100:00100000:3.0:1789762839.181902:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762839.181953:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067403904 00000100:00100000:0.0:1789762839.181962:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067403904 00000100:00100000:0.0:1789762839.181998:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312423499049 time 312423584244 state 0x0 ref 1 00000100:00100000:0.0:1789762839.182009:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312523584244 time 312423584244 state 0x1 ref 1 00000100:00100000:0.0:1789762839.276646:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312523584244 time 312523711757 state 0x0 ref 1 00000100:00100000:0.0:1789762839.276673:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067403904:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762839.276719:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e2d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.276737:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762839.276743:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762839.276748:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e2d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.276759:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e2d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.279244:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e2d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.279254:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e2d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.279259:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e2d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.279264:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ffb8fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762839.279268:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e2d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.279273:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e2d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.279286:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e2d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.279291:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e2d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762839.279338:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067403904:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2677us (97437us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762839.279345:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 437 00000100:00100000:0.0:1789762839.279359:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312523584244 time 312523711757 state 0x2 ref 0 00000100:00100000:2.0:1789762839.281270:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762839.281318:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067404032 00000100:00100000:1.0:1789762839.281326:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067404032 00000100:00100000:1.0:1789762839.281361:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312523584244 time 312523711757 state 0x0 ref 1 00000100:00100000:1.0:1789762839.281372:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312623711757 time 312523711757 state 0x1 ref 1 00000100:00100000:1.0:1789762839.376740:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312623711757 time 312623805255 state 0x0 ref 1 00000100:00100000:1.0:1789762839.376771:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067404032:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762839.376815:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e34 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.376830:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762839.376833:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762839.376838:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e34 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.376848:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e34 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.380174:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e34 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.380185:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e34 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.380192:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e34 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.380206:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762839.380210:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e34 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.380219:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e34 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.380234:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e34 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.380240:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080005955/0xdf5a7ac59f175e34 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762839.380286:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067404032:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3530us (99017us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762839.380293:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 438 00000100:00100000:0.0:1789762839.380315:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312623711757 time 312623805255 state 0x2 ref 0 00000100:00100000:2.0:1789762839.382249:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762839.382302:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067404160 00000100:00100000:1.0:1789762839.382310:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067404160 00000100:00100000:1.0:1789762839.382345:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312623711757 time 312623805255 state 0x0 ref 1 00000100:00100000:1.0:1789762839.382355:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312723805255 time 312623805255 state 0x1 ref 1 00000100:00100000:1.0:1789762839.476813:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312723805255 time 312723876807 state 0x0 ref 1 00000100:00100000:1.0:1789762839.476838:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff617800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067404160:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762839.476875:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e3b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.476889:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762839.476893:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762839.476897:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e3b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.476907:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e3b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.479716:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e3b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.479727:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e3b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.479733:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e3b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.479738:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762839.479742:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e3b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.479748:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e3b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.479761:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e3b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.479766:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e3b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762839.479814:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff617800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067404160:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2987us (97567us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762839.479821:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 439 00000100:00100000:1.0:1789762839.479829:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312723805255 time 312723876807 state 0x2 ref 0 00000100:00100000:2.0:1789762839.481703:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762839.481763:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067404288 00000100:00100000:0.0:1789762839.481772:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067404288 00000100:00100000:0.0:1789762839.481814:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312723805255 time 312723876807 state 0x0 ref 1 00000100:00100000:0.0:1789762839.481829:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312823876807 time 312723876807 state 0x1 ref 1 00000100:00100000:0.0:1789762839.576875:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312823876807 time 312823941252 state 0x0 ref 1 00000100:00100000:0.0:1789762839.576900:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067404288:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762839.576960:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e42 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.576974:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762839.576978:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762839.576982:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e42 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.576993:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e42 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.580022:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e42 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.580037:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e42 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.580050:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e42 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.580060:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ffb8fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762839.580066:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e42 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.580078:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e42 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.580099:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e42 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.580110:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e42 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762839.580181:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067404288:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3287us (98476us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762839.580192:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 440 00000100:00100000:1.0:1789762839.580203:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312823876807 time 312823941252 state 0x2 ref 0 00000100:00100000:2.0:1789762839.582165:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762839.582210:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067404416 00000100:00100000:0.0:1789762839.582219:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067404416 00000100:00100000:0.0:1789762839.582258:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312823876807 time 312823941252 state 0x0 ref 1 00000100:00100000:0.0:1789762839.582273:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312923941252 time 312823941252 state 0x1 ref 1 00000100:00100000:0.0:1789762839.676980:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312923941252 time 312924044449 state 0x0 ref 1 00000100:00100000:0.0:1789762839.677006:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff617100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067404416:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762839.677044:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e49 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.677058:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762839.677062:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762839.677067:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e49 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.677078:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e49 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.679974:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e49 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.679983:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e49 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.679989:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e49 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.679994:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ffb8fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762839.679997:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e49 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.680012:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e49 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.680025:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e49 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.680030:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e49 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762839.680079:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff617100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067404416:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3084us (97914us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762839.680086:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 441 00000100:00100000:0.0:1789762839.680093:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312923941252 time 312924044449 state 0x2 ref 0 00000100:00100000:3.0:1789762839.682004:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762839.682073:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067404544 00000100:00100000:1.0:1789762839.682083:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067404544 00000100:00100000:1.0:1789762839.682119:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 312923941252 time 312924044449 state 0x0 ref 1 00000100:00100000:1.0:1789762839.682132:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313024044449 time 312924044449 state 0x1 ref 1 00000100:00100000:1.0:1789762839.777037:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313024044449 time 313024103532 state 0x0 ref 1 00000100:00100000:1.0:1789762839.777060:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067404544:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762839.777097:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e50 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.777114:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762839.777119:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762839.777125:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e50 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.777144:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e50 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.779986:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e50 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.779997:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e50 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.780005:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e50 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.780015:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762839.780019:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e50 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.780026:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e50 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.780039:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e50 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.780044:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e50 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762839.780088:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067404544:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3036us (98091us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762839.780095:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 442 00000100:00100000:1.0:1789762839.780104:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313024044449 time 313024103532 state 0x2 ref 0 00000100:00100000:3.0:1789762839.781908:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762839.781966:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067404672 00000100:00100000:1.0:1789762839.781974:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067404672 00000100:00100000:1.0:1789762839.782008:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313024044449 time 313024103532 state 0x0 ref 1 00000100:00100000:1.0:1789762839.782019:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313124103532 time 313024103532 state 0x1 ref 1 00000100:00100000:1.0:1789762839.877120:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313124103532 time 313124184320 state 0x0 ref 1 00000100:00100000:1.0:1789762839.877147:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067404672:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762839.877184:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e57 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.877197:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762839.877201:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762839.877206:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e57 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.877217:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e57 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.880011:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e57 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.880020:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e57 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.880026:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e57 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.880031:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f7d93c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762839.880034:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e57 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.880041:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e57 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.880055:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e57 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.880060:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b83a4c4/0xdf5a7ac59f175e57 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762839.880105:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067404672:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2970us (98198us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762839.880112:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 443 00000100:00100000:1.0:1789762839.880120:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313124103532 time 313124184320 state 0x2 ref 0 00000100:00100000:2.0:1789762839.882253:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762839.882306:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067404800 00000100:00100000:0.0:1789762839.882314:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067404800 00000100:00100000:0.0:1789762839.882351:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313124103532 time 313124184320 state 0x0 ref 1 00000100:00100000:0.0:1789762839.882362:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313224184320 time 313124184320 state 0x1 ref 1 00000100:00100000:0.0:1789762839.977211:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313224184320 time 313224277582 state 0x0 ref 1 00000100:00100000:0.0:1789762839.977236:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067404800:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762839.977283:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e5e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.977320:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762839.977325:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762839.977332:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e5e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.977356:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e5e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.980269:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e5e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.980280:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e5e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.980286:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e5e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.980291:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ffb8fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762839.980295:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e5e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.980305:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e5e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.980319:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e5e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.980336:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e5e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762839.980382:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067404800:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3156us (98131us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762839.980389:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 444 00000100:00100000:0.0:1789762839.980397:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313224184320 time 313224277582 state 0x2 ref 0 00000100:00100000:2.0:1789762839.982236:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762839.982294:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067404928 00000100:00100000:1.0:1789762839.982303:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067404928 00000100:00100000:1.0:1789762839.982343:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313224184320 time 313224277582 state 0x0 ref 1 00000100:00100000:1.0:1789762839.982358:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313324277582 time 313224277582 state 0x1 ref 1 00000100:00100000:0.0:1789762840.077319:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313324277582 time 313324382817 state 0x0 ref 1 00000100:00100000:0.0:1789762840.077345:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067404928:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762840.077382:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e65 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.077400:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762840.077405:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762840.077412:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e65 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.077425:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e65 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.080431:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e65 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.080455:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e65 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.080462:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e65 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.080467:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ffb8fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762840.080471:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e65 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.080477:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e65 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.080491:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e65 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.080503:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e65 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762840.080565:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067404928:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3229us (98329us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762840.080573:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 445 00000100:00100000:0.0:1789762840.080581:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313324277582 time 313324382817 state 0x2 ref 0 00000100:00100000:3.0:1789762840.082354:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762840.082398:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067405056 00000100:00100000:1.0:1789762840.082406:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067405056 00000100:00100000:1.0:1789762840.082441:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313324277582 time 313324382817 state 0x0 ref 1 00000100:00100000:1.0:1789762840.082451:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313424382817 time 313324382817 state 0x1 ref 1 00000100:00100000:1.0:1789762840.177385:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313424382817 time 313424449334 state 0x0 ref 1 00000100:00100000:1.0:1789762840.177413:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067405056:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762840.177449:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000091ef13d6/0xdf5a7ac59f175e6c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.177463:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.177466:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.177471:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000091ef13d6/0xdf5a7ac59f175e6c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.177483:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000091ef13d6/0xdf5a7ac59f175e6c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.180247:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000091ef13d6/0xdf5a7ac59f175e6c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.180274:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000091ef13d6/0xdf5a7ac59f175e6c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.180284:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000091ef13d6/0xdf5a7ac59f175e6c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.180293:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bd2600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762840.180298:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000091ef13d6/0xdf5a7ac59f175e6c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.180309:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000091ef13d6/0xdf5a7ac59f175e6c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.180328:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000091ef13d6/0xdf5a7ac59f175e6c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.180337:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000091ef13d6/0xdf5a7ac59f175e6c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762840.180393:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067405056:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2989us (98038us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762840.180403:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 446 00000100:00100000:0.0:1789762840.180413:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313424382817 time 313424449334 state 0x2 ref 0 00000100:00100000:3.0:1789762840.182505:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762840.182560:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067405184 00000100:00100000:0.0:1789762840.182574:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067405184 00000100:00100000:0.0:1789762840.182607:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313424382817 time 313424449334 state 0x0 ref 1 00000100:00100000:0.0:1789762840.182619:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313524449334 time 313424449334 state 0x1 ref 1 00010000:00010000:2.0:1789762840.192022:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762840.277440:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313524449334 time 313524506336 state 0x0 ref 1 00000100:00100000:0.0:1789762840.277465:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067405184:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762840.277503:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e73 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.277518:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762840.277522:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762840.277527:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e73 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.277539:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e73 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.280441:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e73 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.280452:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e73 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.280458:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e73 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.280473:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ffb8fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762840.280477:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e73 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.280483:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e73 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.280496:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e73 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.280501:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e73 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762840.280564:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067405184:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3108us (98059us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762840.280571:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 447 00000100:00100000:0.0:1789762840.280582:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313524449334 time 313524506336 state 0x2 ref 0 00000100:00100000:3.0:1789762840.282599:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762840.282634:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067405312 00000100:00100000:0.0:1789762840.282641:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067405312 00000100:00100000:0.0:1789762840.282675:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313524449334 time 313524506336 state 0x0 ref 1 00000100:00100000:0.0:1789762840.282686:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313624506336 time 313524506336 state 0x1 ref 1 00000100:00100000:0.0:1789762840.377501:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313624506336 time 313624565826 state 0x0 ref 1 00000100:00100000:0.0:1789762840.377537:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067405312:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762840.377586:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e7a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.377600:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762840.377603:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762840.377608:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e7a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.377619:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e7a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.380373:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e7a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.380383:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e7a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.380390:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e7a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.380395:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ffb8fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762840.380399:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e7a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.380405:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e7a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.380429:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e7a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.380436:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032e7668b/0xdf5a7ac59f175e7a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762840.380482:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067405312:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2952us (97884us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762840.380489:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 448 00000100:00100000:0.0:1789762840.380497:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313624506336 time 313624565826 state 0x2 ref 0 00000100:00100000:1.0:1789762840.382331:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762840.382367:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067405440 00000100:00100000:0.0:1789762840.382375:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067405440 00000100:00100000:0.0:1789762840.382407:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313624506336 time 313624565826 state 0x0 ref 1 00000100:00100000:0.0:1789762840.382417:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313724565826 time 313624565826 state 0x1 ref 1 00000100:00100000:1.0:1789762840.477616:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313724565826 time 313724680852 state 0x0 ref 1 00000100:00100000:1.0:1789762840.477643:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067405440:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762840.477679:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f175e81 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.477693:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.477697:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.477701:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f175e81 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.477712:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f175e81 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.480646:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f175e81 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.480660:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f175e81 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.480669:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f175e81 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.480696:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762840.480702:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f175e81 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.480713:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f175e81 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.480732:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f175e81 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.480738:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f175e81 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762840.480792:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067405440:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3158us (98459us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762840.480800:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 449 00000100:00100000:0.0:1789762840.480809:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313724565826 time 313724680852 state 0x2 ref 0 00000100:00100000:1.0:1789762840.482736:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762840.482776:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067405568 00000100:00100000:0.0:1789762840.482785:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067405568 00000100:00100000:0.0:1789762840.482827:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313724565826 time 313724680852 state 0x0 ref 1 00000100:00100000:0.0:1789762840.482841:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313824680852 time 313724680852 state 0x1 ref 1 00000100:00100000:1.0:1789762840.577678:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313824680852 time 313824745116 state 0x0 ref 1 00000100:00100000:1.0:1789762840.577699:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067405568:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762840.577733:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e88 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.577748:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.577752:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.577757:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e88 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.577767:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e88 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.580212:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e88 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.580235:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e88 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.580243:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e88 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.580249:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762840.580253:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e88 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.580260:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e88 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.580274:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e88 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.580279:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e88 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762840.580334:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067405568:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2639us (97597us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762840.580346:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 450 00000100:00100000:0.0:1789762840.580358:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313824680852 time 313824745116 state 0x2 ref 0 00000100:00100000:3.0:1789762840.582285:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762840.582327:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067405696 00000100:00100000:0.0:1789762840.582344:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067405696 00000100:00100000:0.0:1789762840.582386:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313824680852 time 313824745116 state 0x0 ref 1 00000100:00100000:0.0:1789762840.582399:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313924745116 time 313824745116 state 0x1 ref 1 00000100:00100000:1.0:1789762840.677799:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313924745116 time 313924860976 state 0x0 ref 1 00000100:00100000:1.0:1789762840.677829:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067405696:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762840.677872:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e8f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.677888:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.677893:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.677898:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e8f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.677912:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e8f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.681047:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e8f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.681059:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e8f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.681068:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e8f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.681077:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762840.681082:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e8f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.681090:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e8f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.681107:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e8f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.681125:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e8f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762840.681185:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067405696:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3367us (98900us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762840.681195:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 451 00000100:00100000:1.0:1789762840.681205:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313924745116 time 313924860976 state 0x2 ref 0 00000100:00100000:3.0:1789762840.683167:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762840.683214:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067405824 00000100:00100000:0.0:1789762840.683224:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067405824 00000100:00100000:0.0:1789762840.683261:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 313924745116 time 313924860976 state 0x0 ref 1 00000100:00100000:0.0:1789762840.683273:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314024860976 time 313924860976 state 0x1 ref 1 00000100:00100000:1.0:1789762840.777873:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314024860976 time 314024938807 state 0x0 ref 1 00000100:00100000:1.0:1789762840.777900:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067405824:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762840.777965:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e96 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.777983:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762840.777988:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.777996:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e96 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.778011:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e96 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.780985:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e96 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.781000:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e96 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.781018:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e96 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.781028:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762840.781033:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e96 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.781044:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e96 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.781058:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e96 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.781064:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e96 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762840.781112:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067405824:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3224us (97947us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762840.781119:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 452 00000100:00100000:1.0:1789762840.781127:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314024860976 time 314024938807 state 0x2 ref 0 00000100:00100000:3.0:1789762840.782993:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762840.783047:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067405952 00000100:00100000:0.0:1789762840.783056:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067405952 00000100:00100000:0.0:1789762840.783102:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314024860976 time 314024938807 state 0x0 ref 1 00000100:00100000:0.0:1789762840.783123:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314124938807 time 314024938807 state 0x1 ref 1 00000100:00100000:1.0:1789762840.877997:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314124938807 time 314125061242 state 0x0 ref 1 00000100:00100000:1.0:1789762840.878023:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067405952:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762840.878060:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e9d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.878073:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.878077:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.878081:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e9d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.878092:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e9d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.881224:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e9d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.881235:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e9d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.881242:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e9d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.881247:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762840.881251:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e9d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.881257:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e9d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.881274:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e9d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.881279:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175e9d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762840.881327:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067405952:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3315us (98336us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762840.881334:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 453 00000100:00100000:1.0:1789762840.881342:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314124938807 time 314125061242 state 0x2 ref 0 00000100:00100000:2.0:1789762840.883403:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762840.883469:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067406080 00000100:00100000:1.0:1789762840.883478:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067406080 00000100:00100000:1.0:1789762840.883512:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314124938807 time 314125061242 state 0x0 ref 1 00000100:00100000:1.0:1789762840.883522:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314225061242 time 314125061242 state 0x1 ref 1 00000100:00100000:1.0:1789762840.978072:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314225061242 time 314225138172 state 0x0 ref 1 00000100:00100000:1.0:1789762840.978094:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067406080:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762840.978129:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ea4 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.978143:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.978147:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.978151:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ea4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.978163:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ea4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.981100:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ea4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.981117:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ea4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.981125:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ea4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.981130:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762840.981134:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ea4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.981140:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ea4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.981155:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ea4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.981161:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ea4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762840.981227:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067406080:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3140us (97823us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762840.981234:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 454 00000100:00100000:1.0:1789762840.981242:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314225061242 time 314225138172 state 0x2 ref 0 00000100:00100000:2.0:1789762840.983421:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762840.983459:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067406208 00000100:00100000:1.0:1789762840.983467:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067406208 00000100:00100000:1.0:1789762840.983501:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314225061242 time 314225138172 state 0x0 ref 1 00000100:00100000:1.0:1789762840.983512:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314325138172 time 314225138172 state 0x1 ref 1 00000100:00100000:3.0:1789762841.078194:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314325138172 time 314325258966 state 0x0 ref 1 00000100:00100000:3.0:1789762841.078231:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067406208:12345-192.168.203.37@tcp:3:dd.0 00000100:00080000:1.0:1789762841.078255:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762841.078269:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175eab lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.078283:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762841.078287:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762841.078292:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175eab lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.078303:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175eab lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.080995:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175eab lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.081005:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175eab lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.081023:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175eab lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.081028:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f1600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762841.081032:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175eab lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.081038:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175eab lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.081050:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175eab lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.081056:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ec6baa0/0xdf5a7ac59f175eab lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762841.081101:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067406208:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2892us (97681us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762841.081107:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 455 00000100:00100000:3.0:1789762841.081115:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314325138172 time 314325258966 state 0x2 ref 0 00000100:00100000:3.0:1789762841.082849:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762841.082899:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067406336 00000100:00100000:1.0:1789762841.082907:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067406336 00000100:00100000:1.0:1789762841.082959:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314325138172 time 314325258966 state 0x0 ref 1 00000100:00100000:1.0:1789762841.082970:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314425258966 time 314325258966 state 0x1 ref 1 00000100:00100000:1.0:1789762841.088055:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f58000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074682880:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762841.088093:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762841.088138:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074682880 00000100:00100000:0.0:1789762841.088146:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074682880 00000100:00100000:0.0:1789762841.088186:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 290 00000100:00100000:0.0:1789762841.088193:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074682880:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762841.088219:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762841.088259:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074682880:12345-0@lo:41:osp-pre-1-0.0 Request processed in 68us (165us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762841.088265:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 290 00000100:00100000:1.0:1789762841.088301:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f58000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074682880:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762841.178267:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314425258966 time 314425333082 state 0x0 ref 1 00000100:00100000:1.0:1789762841.178294:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067406336:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762841.178330:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175eb2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.178343:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762841.178347:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762841.178351:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175eb2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.178362:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175eb2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.180884:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175eb2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.180895:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175eb2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.180902:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175eb2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.180908:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762841.180911:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175eb2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.180934:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175eb2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.180954:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175eb2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.180967:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175eb2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762841.181018:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067406336:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2736us (98169us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762841.181026:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 456 00000100:00100000:1.0:1789762841.181033:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314425258966 time 314425333082 state 0x2 ref 0 00000100:00100000:0.0:1789762841.182898:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762841.182948:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067406464 00000100:00100000:1.0:1789762841.182956:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067406464 00000100:00100000:1.0:1789762841.182990:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314425258966 time 314425333082 state 0x0 ref 1 00000100:00100000:1.0:1789762841.183008:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314525333082 time 314425333082 state 0x1 ref 1 00010000:00010000:1.0:1789762841.216123:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762841.278382:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314525333082 time 314525447071 state 0x0 ref 1 00000100:00100000:0.0:1789762841.278420:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067406464:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762841.278454:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175eb9 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.278468:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762841.278472:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762841.278477:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175eb9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.278488:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175eb9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.281483:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175eb9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.281499:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175eb9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.281510:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175eb9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.281518:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762841.281522:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175eb9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.281529:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175eb9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.281563:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175eb9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.281569:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175eb9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762841.281615:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067406464:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3201us (98717us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762841.281623:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 457 00000100:00100000:0.0:1789762841.281631:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314525333082 time 314525447071 state 0x2 ref 0 00000100:00100000:0.0:1789762841.283832:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762841.283875:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067406592 00000100:00100000:3.0:1789762841.283886:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067406592 00000100:00100000:3.0:1789762841.283944:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314525333082 time 314525447071 state 0x0 ref 1 00000100:00100000:3.0:1789762841.283961:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314625447071 time 314525447071 state 0x1 ref 1 00000100:00100000:0.0:1789762841.378532:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314625447071 time 314625597601 state 0x0 ref 1 00000100:00100000:0.0:1789762841.378586:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067406592:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762841.378623:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175ec0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.378636:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762841.378640:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762841.378644:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175ec0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.378656:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175ec0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.381465:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175ec0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.381477:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175ec0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.381485:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175ec0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.381491:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762841.381496:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175ec0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.381502:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175ec0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.381517:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175ec0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.381523:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f175ec0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762841.381583:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067406592:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3018us (97751us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762841.381590:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 458 00000100:00100000:1.0:1789762841.381598:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314625447071 time 314625597601 state 0x2 ref 0 00000100:00100000:3.0:1789762841.383467:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762841.383504:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067406720 00000100:00100000:1.0:1789762841.383512:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067406720 00000100:00100000:1.0:1789762841.383559:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314625447071 time 314625597601 state 0x0 ref 1 00000100:00100000:1.0:1789762841.383570:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314725597601 time 314625597601 state 0x1 ref 1 00000100:00100000:0.0:1789762841.478637:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314725597601 time 314725702171 state 0x0 ref 1 00000100:00100000:0.0:1789762841.478664:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067406720:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762841.478702:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ec7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.478717:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762841.478720:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762841.478725:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ec7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.478736:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ec7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.481752:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ec7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.481765:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ec7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.481773:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ec7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.481778:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762841.481790:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ec7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.481798:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ec7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.481812:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ec7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.481817:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ec7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762841.481864:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067406720:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3210us (98398us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762841.481870:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 459 00000100:00100000:0.0:1789762841.481877:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314725597601 time 314725702171 state 0x2 ref 0 00000100:00100000:1.0:1789762841.483684:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762841.483723:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067406848 00000100:00100000:0.0:1789762841.483731:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067406848 00000100:00100000:0.0:1789762841.483775:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314725597601 time 314725702171 state 0x0 ref 1 00000100:00100000:0.0:1789762841.483789:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314825702171 time 314725702171 state 0x1 ref 1 00000100:00100000:1.0:1789762841.578751:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314825702171 time 314825816443 state 0x0 ref 1 00000100:00100000:1.0:1789762841.578780:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067406848:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762841.578818:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ece lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.578831:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762841.578835:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762841.578840:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ece lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.578851:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ece lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.581808:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ece lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.581820:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ece lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.581838:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ece lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.581846:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762841.581859:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ece lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.581871:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ece lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.581889:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ece lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.581898:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f175ece lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762841.581998:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067406848:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3228us (98314us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762841.582009:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 460 00000100:00100000:0.0:1789762841.582020:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314825702171 time 314825816443 state 0x2 ref 0 00000100:00100000:1.0:1789762841.583966:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762841.583999:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067406976 00000100:00100000:0.0:1789762841.584008:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067406976 00000100:00100000:0.0:1789762841.584044:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314825702171 time 314825816443 state 0x0 ref 1 00000100:00100000:0.0:1789762841.584065:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314925816443 time 314825816443 state 0x1 ref 1 00000100:00100000:0.0:1789762841.678848:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314925816443 time 314925912621 state 0x0 ref 1 00000100:00100000:0.0:1789762841.678873:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067406976:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762841.678910:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ed5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.678955:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762841.678958:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762841.678963:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ed5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.678974:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ed5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.681597:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ed5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.681612:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ed5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.681632:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ed5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.681653:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762841.681657:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ed5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.681664:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ed5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.681678:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ed5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.681684:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f175ed5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762841.681730:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067406976:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2867us (97764us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762841.681737:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 461 00000100:00100000:0.0:1789762841.681745:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314925816443 time 314925912621 state 0x2 ref 0 00000100:00100000:2.0:1789762841.706335:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762841.706384:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067407104 00000100:00100000:3.0:1789762841.706394:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067407104 00000100:00100000:3.0:1789762841.706448:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 43 00000100:00100000:3.0:1789762841.706457:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:6823:x1876702067407104:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762841.706470:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762841.706502:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de12dad1/0xdf5a7ac59f175c0b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7d0 expref: 6 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.706525:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762841.706593:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:6823:x1876702067407104:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 137us (266us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762841.706602:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 43 00000100:00100000:2.0:1789762841.716274:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762841.716334:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067407232 00000100:00100000:1.0:1789762841.716343:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067407232 00000100:00080000:2.0:1789762841.716395:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762841.716401:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 242 00000100:00100000:1.0:1789762841.716410:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:13087:x1876702067407232:12345-192.168.203.37@tcp:101:bash.0 00010000:00010000:1.0:1789762841.716427:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762841.716441:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f175edc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7d7 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.716493:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f175ee3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.716504:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f175ee3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.716513:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f175ee3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762841.716554:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 16384 00000004:00200000:1.0:1789762841.716557:0:6420:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:1.0:1789762841.716572:0:6420:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f175ee3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.716591:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f175edc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7d7 expref: 11 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.716603:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f175ee3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7d7 expref: 10 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.716609:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:1.0:1789762841.716612:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762841.716614:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5933800, rc 0) 00000100:00100000:1.0:1789762841.716654:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:13087:x1876702067407232:12345-192.168.203.37@tcp:101:bash.0 Request processed in 246us (388us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762841.716659:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 242 00000100:00100000:1.0:1789762841.721137:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762841.721188:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067407488 00000100:00100000:2.0:1789762841.721195:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067407488 00000100:00100000:2.0:1789762841.721234:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 243 00000100:00100000:2.0:1789762841.721243:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:14027:x1876702067407488:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:2.0:1789762841.721255:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762841.721266:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7200/0xdf5a7ac59f175eea lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7de expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762841.721318:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 16384 00010000:00010000:2.0:1789762841.721460:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7200/0xdf5a7ac59f175eea lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7de expref: 12 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.721473:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762841.721476:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762841.721481:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7200/0xdf5a7ac59f175eea lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7de expref: 12 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.721492:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc7200, rc 0) 00000100:00100000:2.0:1789762841.721561:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:14027:x1876702067407488:12345-192.168.203.37@tcp:101:rm.0 Request processed in 318us (427us total) trans 4294967398 rc 301/301 00000100:00100000:2.0:1789762841.721568:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 243 00000100:00100000:2.0:1789762841.723197:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762841.723235:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067407616 00000100:00100000:1.0:1789762841.723240:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067407616 00000100:00100000:1.0:1789762841.723276:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 65 00000100:00100000:1.0:1789762841.723287:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:14027:x1876702067407616:12345-192.168.203.37@tcp:37:rm.0 00000100:00100000:1.0:1789762841.725197:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:14027:x1876702067407616:12345-192.168.203.37@tcp:37:rm.0 Request processed in 1912us (2000us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762841.725207:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 65 00000100:00100000:3.0:1789762841.726453:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762841.726492:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067407872 00000100:00100000:0.0:1789762841.726497:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067407872 00000100:00100000:0.0:1789762841.726523:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 66 00000100:00100000:0.0:1789762841.726540:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:14027:x1876702067407872:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:0.0:1789762841.726642:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:14027:x1876702067407872:12345-192.168.203.37@tcp:35:rm.0 Request processed in 112us (189us total) trans 4294967399 rc 0/0 00000100:00100000:0.0:1789762841.726656:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 66 00000100:00100000:2.0:1789762841.727425:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762841.727466:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067408128 00000100:00100000:1.0:1789762841.727471:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067408128 00000100:00100000:1.0:1789762841.727497:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 244 00000100:00100000:1.0:1789762841.727503:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:14027:x1876702067408128:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:1.0:1789762841.727513:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762841.727523:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f175ef8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7e5 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.727581:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f175eff lrc: 3/1,0 mode: --/CR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.727591:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f175eff lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x12:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.727599:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f175eff lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x12:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762841.727608:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 16384 00010000:00010000:1.0:1789762841.727671:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f175ef8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7e5 expref: 15 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.727682:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f175eff lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x12:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7e5 expref: 14 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.727688:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:1.0:1789762841.727690:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1789762841.727692:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5933c00, rc 0) 00000100:00100000:1.0:1789762841.727719:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14027:x1876702067408128:12345-192.168.203.37@tcp:101:rm.0 Request processed in 217us (297us total) trans 4294967400 rc 0/0 00000100:00100000:1.0:1789762841.727725:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 244 00000100:00100000:2.0:1789762841.728524:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762841.728570:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067408256 00000100:00100000:0.0:1789762841.728575:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067408256 00000100:00100000:0.0:1789762841.728600:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 245 00000100:00100000:0.0:1789762841.728606:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14027:x1876702067408256:12345-192.168.203.37@tcp:41:rm.0 00000004:00000024:0.0:1789762841.728636:0:6418:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7614758912 avail 7596933120 00000100:00100000:0.0:1789762841.728667:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14027:x1876702067408256:12345-192.168.203.37@tcp:41:rm.0 Request processed in 61us (144us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762841.728672:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 245 00000100:00100000:3.0:1789762841.729434:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762841.729465:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067408512 00000100:00100000:1.0:1789762841.729470:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067408512 00000100:00100000:1.0:1789762841.729494:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 67 00000100:00100000:1.0:1789762841.729500:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14027:x1876702067408512:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:1.0:1789762841.729684:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:14027:x1876702067408512:12345-192.168.203.37@tcp:35:rm.0 Request processed in 184us (251us total) trans 4294967401 rc 0/0 00000100:00100000:1.0:1789762841.729689:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 67 00000100:00100000:2.0:1789762841.730398:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762841.730438:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067408768 00000100:00100000:3.0:1789762841.730443:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067408768 00000100:00100000:3.0:1789762841.730470:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 246 00000100:00100000:3.0:1789762841.730476:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:14027:x1876702067408768:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:3.0:1789762841.730500:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1789762841.730513:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37f7d92400/0xdf5a7ac59f175bfd lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7c9 expref: 15 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.730552:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6200/0xdf5a7ac59f17591e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7c2 expref: 14 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.730560:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff9c37d3beef00) 00010000:00010000:3.0:1789762841.730569:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f175ee3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7d7 expref: 13 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.730575:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:3.0:1789762841.730577:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762841.730587:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f0d lrc: 3/0,1 mode: --/CW res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.730593:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f0d lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.730598:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f0d lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.730608:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f175f14 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.730612:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f175f14 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.730616:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f175f14 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.730849:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d600/0xdf5a7ac59f175f1b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.730862:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d600/0xdf5a7ac59f175f1b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.730868:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d600/0xdf5a7ac59f175f1b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789762841.731750:0:7744:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].6, 64 off8576 00000004:00080000:3.0:1789762841.731760:0:7744:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 314978841075ns, next at 315978838773ns, rc = 0 00010000:00010000:3.0:1789762841.731793:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f175f22 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731800:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f175f22 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731805:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f175f22 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731811:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f175f22 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731816:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f175f22 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731820:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f175f22 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731835:0:7744:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762841.731844:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d600/0xdf5a7ac59f175f1b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731848:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d600/0xdf5a7ac59f175f1b lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731858:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d600/0xdf5a7ac59f175f1b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.731862:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762841.731863:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762841.731868:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d600/0xdf5a7ac59f175f1b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.731873:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f175f22 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731875:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d600/0xdf5a7ac59f175f1b lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.731892:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762841.731894:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff9c37c72bae00) 00010000:00010000:3.0:1789762841.731898:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789762841.731901:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d600/0xdf5a7ac59f175f1b lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.731903:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f175f22 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731907:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d600/0xdf5a7ac59f175f1b lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.731939:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f175f22 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731943:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f0d lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.731951:0:6407:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff9c37c72bae00) 00010000:00010000:3.0:1789762841.731951:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f0d lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.731959:0:6407:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789762841.731962:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f0d lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.731964:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f175f22 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731972:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f0d lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.731975:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f175f22 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.731980:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762841.731989:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f0d lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.731992:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762841.731995:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f0d lrc: 5/0,0 mode: CW/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.731996:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762841.731999:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762841.732008:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:3.0:1789762841.732010:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762841.732013:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f0d lrc: 3/0,0 mode: --/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.732020:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f0d lrc: 0/0,0 mode: --/CW res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762841.732032:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009802e337 reply state 00000000ff876120 transno 4294967402 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f175f14 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.732044:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f175f14 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.732052:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x4009f790 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762841.732054:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1789762841.732057:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762841.732094:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14027:x1876702067408768:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1618us (1697us total) trans 4294967402 rc 0/0 00000100:00100000:3.0:1789762841.732100:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 246 00000100:00100000:2.0:1789762841.733083:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762841.733122:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067409024 00000100:00100000:0.0:1789762841.733127:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067409024 00000100:00100000:0.0:1789762841.733152:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 68 00000100:00100000:0.0:1789762841.733159:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14027:x1876702067409024:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:0.0:1789762841.733241:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:14027:x1876702067409024:12345-192.168.203.37@tcp:35:rm.0 Request processed in 82us (158us total) trans 4294967403 rc 0/0 00000100:00100000:0.0:1789762841.733246:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 68 00000100:00100000:1.0:1789762841.733962:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762841.733993:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067409280 00000100:00100000:0.0:1789762841.733997:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067409280 00000100:00100000:0.0:1789762841.734021:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 247 00000100:00100000:0.0:1789762841.734027:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:14027:x1876702067409280:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:0.0:1789762841.734046:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1789762841.734057:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f175eff lrc: 0/0,0 mode: --/CR res: [0x200000402:0x12:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7e5 expref: 15 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734073:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230000/0xdf5a7ac59f175902 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7bb expref: 14 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734095:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37c72ba700) 00010000:00010000:0.0:1789762841.734109:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d9230200/0xdf5a7ac59f1758a0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7a6 expref: 13 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734116:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762841.734118:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762841.734128:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175f29 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734134:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175f29 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734139:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175f29 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734147:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f175f30 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734152:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f175f30 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734157:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f175f30 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734333:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175f37 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734339:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175f37 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734351:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175f37 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762841.734739:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 16384 00010000:00010000:0.0:1789762841.734747:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175f37 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734752:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175f37 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734756:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175f37 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734759:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762841.734762:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175f37 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734766:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175f37 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734772:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff9c37e125a800) 00010000:00010000:0.0:1789762841.734774:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762841.734776:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175f37 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734780:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175f37 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734791:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175f29 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734795:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175f29 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734803:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175f29 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734808:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175f29 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734811:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762841.734813:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175f29 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734816:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175f29 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734821:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762841.734823:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762841.734824:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175f29 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734828:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175f29 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762841.734833:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000042567ac7 reply state 000000004f91b20a transno 4294967404 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f175f30 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734839:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f175f30 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.734843:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37e125ab00) 00010000:00010000:0.0:1789762841.734845:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762841.734846:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762841.734878:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14027:x1876702067409280:12345-192.168.203.37@tcp:36:rm.0 Request processed in 851us (917us total) trans 4294967404 rc 0/0 00000100:00100000:0.0:1789762841.734889:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 247 00000100:00100000:3.0:1789762842.112059:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074683008:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762842.112128:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762842.112184:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074683008 00000100:00100000:0.0:1789762842.112193:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074683008 00000100:00100000:0.0:1789762842.112254:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 291 00000100:00100000:0.0:1789762842.112270:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074683008:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762842.112311:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:0.0:1789762842.112376:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074683008:12345-0@lo:41:osp-pre-0-1.0 Request processed in 112us (246us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762842.112390:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 291 00000100:00100000:3.0:1789762842.112407:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074683008:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1789762842.239999:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1789762842.570004:0:15960:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 315817083504ns, next at 315978838773ns, rc = 0 00000004:00080000:0.0:1789762842.577635:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 10 committed 00000001:00080000:0.0:1789762842.577664:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967404 is committed 00010000:00010000:1.0:1789762842.577682:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f175f14 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.577693:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f175f14 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762842.577698:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 9 committed 00010000:00010000:1.0:1789762842.577700:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f175f14 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.577706:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273ce00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762842.577710:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f175f14 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.577716:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f175f14 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.577730:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x4009f790 (ffff9c37c72ba500) 00010000:00010000:1.0:1789762842.577734:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762842.577737:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f175f14 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.577742:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f175f14 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x12:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762842.577749:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074683136:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1789762842.577762:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f175f30 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.577767:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f175f30 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.577775:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f175f30 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.577779:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762842.577781:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f175f30 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.577786:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f175f30 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.577792:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37e125ab00) 00010000:00010000:1.0:1789762842.577793:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762842.577796:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f175f30 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.577800:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f175f30 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762842.577809:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762842.577834:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074683136 00000100:00100000:2.0:1789762842.577841:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074683136 00000100:00080000:1.0:1789762842.577872:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762842.577891:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:2.0:1789762842.577901:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074683136:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1789762842.577947:0:8052:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:6 count 1 00002000:00010000:2.0:1789762842.577971:0:8052:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x6:0x0] initial LVB size: 104857600, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x3200b 00010000:00010000:2.0:1789762842.577977:0:8052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175f3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762842.577988:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=315s v=(0 0) 00010000:00010000:2.0:1789762842.577992:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762842.577997:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175f3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762842.578008:0:8052:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175f3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762842.578014:0:8052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175f3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762842.578025:0:8052:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175f3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762842.578038:0:8052:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762842.578051:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762842.578060:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175f3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.578073:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762842.578077:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175f3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.578088:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175f3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.578113:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175f3e lrc: 1/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.578117:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f175f3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.578129:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762842.578130:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762842.578132:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762842.578407:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074683136:12345-0@lo:6:osp-syn-0-0.0 Request processed in 508us (600us total) trans 4294967793 rc 0/0 00000100:00100000:2.0:1789762842.578415:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000004:00080000:0.0:1789762842.578439:0:3427:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37f7f5b480/2, rc 0, transno 4294967793 00000100:00100000:0.0:1789762842.578452:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074683136:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1789762842.578569:0:15960:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 315825649984ns, next at 303048151596ns, rc = 0 00000004:00080000:2.0:1789762842.578636:0:15960:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 315825717334ns, next at 0ns, rc = 0 00000004:00080000:2.0:1789762842.578741:0:15960:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 315825822599ns, next at 303048116981ns, rc = 0 00000004:00080000:1.0:1789762842.579769:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 5 committed 00000004:00080000:1.0:1789762842.585972:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 14 committed 00000004:00080000:3.0:1789762842.586608:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 14 committed 00000100:00100000:2.0:1789762843.080003:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762843.080032:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762843.080056:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762843.080079:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067409536 00000100:00100000:2.0:1789762843.080081:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762843.080090:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067409536 00000100:00100000:1.0:1789762843.080103:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067409408 00000100:00100000:2.0:1789762843.080109:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762843.080112:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067409408 00000100:00100000:0.0:1789762843.080119:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067409664 00000100:00100000:0.0:1789762843.080124:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067409664 00000100:00100000:2.0:1789762843.080134:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762843.080163:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762843.080183:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 349 00000100:00100000:3.0:1789762843.080189:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067410048 00000100:00100000:0.0:1789762843.080190:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067410176 00000100:00100000:3.0:1789762843.080196:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067410048 00000100:00100000:0.0:1789762843.080197:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067410176 00000100:00100000:1.0:1789762843.080198:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d157bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067409408:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762843.080207:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067409792 00000100:00100000:2.0:1789762843.080212:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067409792 00000100:00100000:3.0:1789762843.080231:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 438 00000100:00100000:0.0:1789762843.080241:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 439 00000100:00100000:3.0:1789762843.080247:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067409536:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762843.080249:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067409920 00000100:00100000:2.0:1789762843.080255:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067409920 00000100:00100000:0.0:1789762843.080259:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9928000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067409664:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762843.080272:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d157bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067409408:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 79us (276us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762843.080285:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 172 00000100:00100000:1.0:1789762843.080286:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 349 00000100:00100000:2.0:1789762843.080295:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067409792:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762843.080311:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067409536:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 71us (274us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762843.080320:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9928000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067409664:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 63us (257us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762843.080325:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 438 00000100:00100000:2.0:1789762843.080333:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067409792:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 39us (251us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762843.080339:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 439 00000100:00100000:3.0:1789762843.080346:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 440 00000100:00100000:2.0:1789762843.080362:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 172 00000100:00100000:3.0:1789762843.080362:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9928380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067410048:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762843.080367:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 441 00000100:00100000:0.0:1789762843.080382:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9928e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067410176:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762843.080389:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 173 00000100:00100000:2.0:1789762843.080402:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067409920:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762843.080404:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9928380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067410048:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 48us (266us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762843.080412:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 440 00000100:00100000:0.0:1789762843.080419:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9928e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067410176:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 39us (254us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762843.080436:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762843.080437:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067409920:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 37us (327us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762843.080443:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762843.080447:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 441 00000100:00100000:2.0:1789762843.080447:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 173 00000100:00080000:3.0:1789762843.080449:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762843.080456:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789762843.264084:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762844.160071:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f60a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074683264:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762844.160080:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074683520:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.160135:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074683648:0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.160153:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762844.160155:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762844.160187:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762844.160209:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074683392:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.160216:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074684160:0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.160221:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074684032:0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.160239:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762844.160269:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762844.160298:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074683904:0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.160299:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074683520 00000100:00100000:1.0:1789762844.160305:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a09500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074684672:0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.160309:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074683520 00000100:00100000:2.0:1789762844.160346:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762844.160352:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762844.160366:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074684160 00000100:00100000:0.0:1789762844.160372:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074684160 00000100:00100000:2.0:1789762844.160386:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074684416:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.160391:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:1.0:1789762844.160399:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074683776:0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.160439:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762844.160446:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 442 00000100:00100000:0.0:1789762844.160467:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074683520:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.160470:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074683264 00000100:00100000:1.0:1789762844.160478:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074683264 00000100:00100000:2.0:1789762844.160492:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a09f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074684928:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.160532:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:1.0:1789762844.160541:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074683264:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762844.160548:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762844.160553:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074683520:12345-0@lo:400:kworker.0 Request processed in 96us (402us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762844.160563:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 442 00002000:00000024:1.0:1789762844.160577:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 13631488 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762844.160588:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074684416 00000100:00100000:0.0:1789762844.160595:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074684416 00000100:00100000:2.0:1789762844.160595:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074683392 00000100:00100000:2.0:1789762844.160602:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074683392 00000100:00100000:1.0:1789762844.160625:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074683264:12345-0@lo:13:osp-pre-1-0.0 Request processed in 84us (474us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762844.160632:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00100000:0.0:1789762844.160644:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:2.0:1789762844.160648:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 350 00000100:00100000:1.0:1789762844.160668:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074683648 00000100:00100000:0.0:1789762844.160669:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a08700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074684160:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.160669:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d157b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074683392:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.160673:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074683648 00000100:00100000:1.0:1789762844.160726:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074683904 00000100:00100000:1.0:1789762844.160729:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074683904 00000100:00100000:2.0:1789762844.160740:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d157b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074683392:12345-0@lo:400:kworker.0 Request processed in 83us (498us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762844.160752:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 350 00000100:00100000:0.0:1789762844.160755:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a08700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074684160:12345-0@lo:400:kworker.0 Request processed in 94us (482us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762844.160774:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:1.0:1789762844.160774:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:1.0:1789762844.160790:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a08000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074683648:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.160804:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:2.0:1789762844.160813:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 293 00000100:00100000:0.0:1789762844.160822:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074684416:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.160828:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074683904:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.160842:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a08000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074683648:12345-0@lo:400:kworker.0 Request processed in 57us (655us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762844.160848:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:1.0:1789762844.160887:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074684672 00000100:00100000:1.0:1789762844.160890:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074684672 00000100:00100000:0.0:1789762844.160897:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074684416:12345-0@lo:400:kworker.0 Request processed in 79us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762844.160898:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074683904:12345-0@lo:400:kworker.0 Request processed in 76us (552us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762844.160945:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074684928 00000100:00100000:1.0:1789762844.160949:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074684928 00000100:00100000:0.0:1789762844.160953:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:2.0:1789762844.160953:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 293 00000100:00100000:1.0:1789762844.160997:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:2.0:1789762844.161009:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:1.0:1789762844.161014:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a09c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074684672:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762844.161018:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762844.161024:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074684928:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762844.161028:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762844.161062:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a09c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074684672:12345-0@lo:400:kworker.0 Request processed in 55us (711us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762844.161073:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00100000:2.0:1789762844.161088:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074684928:12345-0@lo:400:kworker.0 Request processed in 63us (538us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762844.161098:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00100000:1.0:1789762844.161143:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789762844.161153:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c992a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074685312:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762844.161178:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762844.161184:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762844.161210:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f60a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074685184:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762844.161225:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074684032 00000100:00100000:0.0:1789762844.161229:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074684544:0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.161231:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074684032 00000100:00100000:1.0:1789762844.161245:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762844.161287:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762844.161298:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 445 00000100:00100000:1.0:1789762844.161304:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074685312 00000100:00100000:1.0:1789762844.161309:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074685312 00000100:00100000:2.0:1789762844.161318:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074684032:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.161351:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074683520:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.161353:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074685184 00000100:00100000:1.0:1789762844.161360:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074685184 00000100:00100000:2.0:1789762844.161378:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074684032:12345-0@lo:400:kworker.0 Request processed in 71us (208us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762844.161385:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 445 00000100:00100000:0.0:1789762844.161394:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f58000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074685056:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762844.161404:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:1.0:1789762844.161416:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074685312:12345-0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789762844.161419:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762844.161426:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00000024:1.0:1789762844.161446:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439418880 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762844.161460:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762844.161460:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:2.0:1789762844.161479:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074685184:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789762844.161494:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439418880 pending 0 free 3754950656 avail 3752853504 00000100:00100000:1.0:1789762844.161511:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074685312:12345-0@lo:13:osp-pre-0-0.0 Request processed in 99us (328us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762844.161531:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:0.0:1789762844.161535:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074684032:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.161550:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074685056 00000100:00100000:2.0:1789762844.161552:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074685184:12345-0@lo:13:osp-pre-0-1.0 Request processed in 80us (308us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762844.161554:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074685056 00000100:00100000:2.0:1789762844.161563:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1789762844.161589:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074684544 00000100:00100000:1.0:1789762844.161589:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:0.0:1789762844.161597:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074684544 00000100:00100000:1.0:1789762844.161600:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074685056:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789762844.161612:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 13631488 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762844.161626:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074683904:0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.161644:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:1.0:1789762844.161653:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074685056:12345-0@lo:13:osp-pre-1-1.0 Request processed in 55us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762844.161654:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074684416:0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.161659:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074684544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.161670:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00100000:2.0:1789762844.161674:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a09f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074684928:0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.161705:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f60a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074683264:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762844.161726:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c992a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074685312:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762844.161730:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074683392:0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.161737:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074684544:12345-0@lo:400:kworker.0 Request processed in 79us (455us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762844.161746:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f60a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074685184:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762844.161751:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00100000:0.0:1789762844.161795:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762844.161816:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f58000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074685056:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762844.161828:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074683776 00000100:00100000:2.0:1789762844.161832:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074683776 00000100:00100000:0.0:1789762844.161836:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074684288:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.161839:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074684544:0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.161868:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 446 00000100:00100000:2.0:1789762844.161880:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074683776:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.161882:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762844.161913:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074684288 00000100:00100000:1.0:1789762844.161969:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074684288 00000100:00100000:2.0:1789762844.161984:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074683776:12345-0@lo:400:kworker.0 Request processed in 104us (186us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762844.161987:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a08e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074684800:0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.161995:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 446 00000100:00100000:2.0:1789762844.162048:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 447 00000100:00100000:0.0:1789762844.162050:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762844.162066:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074684288:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789762844.162082:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762844.162115:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074684800 00000100:00100000:1.0:1789762844.162119:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074684800 00000100:00100000:0.0:1789762844.162123:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074683648:0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.162124:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074684288:12345-0@lo:400:kworker.0 Request processed in 68us (248us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762844.162138:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 447 00000100:00100000:1.0:1789762844.162158:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:0.0:1789762844.162160:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074684160:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.162175:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074684800:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.162195:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a09500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074684672:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.162235:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074684800:12345-0@lo:400:kworker.0 Request processed in 66us (193us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762844.162244:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:0.0:1789762844.162272:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074683776:0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.162286:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074684288:0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.162305:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a08e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074684800:0@lo:400:kworker.0 00010000:00010000:3.0:1789762844.288039:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762845.312037:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762846.208083:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789762846.208109:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074685440:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762846.208169:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762846.208210:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074685440 00000100:00100000:1.0:1789762846.208218:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074685440 00000100:00100000:1.0:1789762846.208264:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00100000:1.0:1789762846.208273:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074685440:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762846.208306:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762846.208351:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074685440:12345-0@lo:41:osp-pre-1-0.0 Request processed in 81us (182us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762846.208360:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00000100:00100000:0.0:1789762846.208391:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074685440:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1789762846.336066:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762847.232047:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074685568:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789762847.232110:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762847.232167:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074685568 00000100:00100000:1.0:1789762847.232175:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074685568 00000100:00100000:1.0:1789762847.232233:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 295 00000100:00100000:1.0:1789762847.232247:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074685568:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789762847.232286:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789762847.232357:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074685568:12345-0@lo:41:osp-pre-0-1.0 Request processed in 113us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762847.232366:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 295 00000100:00100000:1.0:1789762847.232424:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074685568:0@lo:41:osp-pre-0-1.0 00000001:00080000:0.0:1789762847.249546:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967793 is committed 00010000:00010000:1.0:1789762847.360030:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762848.200047:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762848.200078:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762848.200104:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067410304 00000100:00100000:2.0:1789762848.200108:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762848.200116:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067410304 00000100:00100000:2.0:1789762848.200137:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762848.200156:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067410560 00000100:00100000:3.0:1789762848.200163:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067410560 00000100:00100000:1.0:1789762848.200171:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067410432 00000100:00100000:1.0:1789762848.200178:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067410432 00000100:00100000:0.0:1789762848.200181:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 351 00000100:00100000:2.0:1789762848.200182:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067410688 00000100:00100000:2.0:1789762848.200188:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067410688 00000100:00100000:0.0:1789762848.200196:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d157b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067410304:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762848.200222:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 448 00000100:00100000:1.0:1789762848.200232:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 449 00000100:00100000:2.0:1789762848.200233:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 178 00000100:00100000:3.0:1789762848.200240:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9928700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067410560:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762848.200249:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067410688:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762848.200255:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067410432:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762848.200262:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d157b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067410304:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 67us (218us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762848.200271:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 351 00000100:00100000:3.0:1789762848.200291:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9928700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067410560:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 58us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762848.200299:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 448 00000100:00100000:2.0:1789762848.200309:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067410688:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 62us (170us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762848.200319:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067410432:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 72us (208us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762848.200319:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 178 00000100:00100000:1.0:1789762848.200329:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 449 00000100:00080000:3.0:1789762848.200339:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789762848.200351:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789762848.200355:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762848.203021:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762848.203044:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762848.203064:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762848.203072:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067410816 00000100:00100000:2.0:1789762848.203078:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067410816 00000100:00100000:0.0:1789762848.203097:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067410944 00000100:00080000:1.0:1789762848.203103:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762848.203104:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067410944 00000100:00100000:3.0:1789762848.203111:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067411072 00000100:00100000:3.0:1789762848.203118:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067411072 00000100:00100000:2.0:1789762848.203122:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 156 00000100:00080000:1.0:1789762848.203133:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762848.203134:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2394:x1876702067410816:12345-192.168.203.37@tcp:9:kworker.0 00000100:00080000:1.0:1789762848.203140:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000020:00000080:2.0:1789762848.203156:0:8052:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xdf5a7ac59f1756e0 00000100:00100000:0.0:1789762848.203161:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 450 00000100:00080000:2.0:1789762848.203168:0:8052:0:(import.c:65:import_set_state_nolock()) ffff9c36cba88800 : changing import state from FULL to CLOSED 00000100:00100000:3.0:1789762848.203170:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 451 00000100:00100000:0.0:1789762848.203175:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067410944:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762848.203186:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a08e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067411072:12345-192.168.203.37@tcp:400:kworker.0 00010000:00010000:1.0:1789762848.203197:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00000004:2.0:1789762848.203206:0:8052:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967381 00000100:00100000:0.0:1789762848.203220:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067410944:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 50us (173us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762848.203231:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a08e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067411072:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 48us (162us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762848.203240:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 450 00000100:00100000:3.0:1789762848.203250:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 451 00000100:00100000:2.0:1789762848.203398:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+2:2394:x1876702067410816:12345-192.168.203.37@tcp:9:kworker.0 Request processed in 267us (376us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762848.203407:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 156 00010000:00010000:2.0:1789762848.384028:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762849.280091:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074685952:0@lo:400:kworker.0 00000100:00100000:3.0:1789762849.280134:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074686080:0@lo:400:kworker.0 00000100:00100000:2.0:1789762849.280156:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074685824:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762849.280190:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762849.280201:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762849.280224:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c992b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074686208:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762849.280247:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762849.280276:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074686592:0@lo:400:kworker.0 00000100:00100000:0.0:1789762849.280283:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8a8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074686336:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762849.280292:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f58000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074685696:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762849.280301:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762849.280319:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762849.280348:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074686080 00000100:00100000:0.0:1789762849.280356:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074686080 00000100:00100000:1.0:1789762849.280358:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074686720:0@lo:400:kworker.0 00000100:00100000:3.0:1789762849.280367:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074687104:0@lo:400:kworker.0 00000100:00100000:2.0:1789762849.280375:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762849.280413:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074686592 00000100:00100000:0.0:1789762849.280419:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074686592 00000100:00100000:2.0:1789762849.280429:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a09500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074686464:0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.280441:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762849.280445:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762849.280461:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074687104 00000100:00100000:0.0:1789762849.280464:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074687104 00000100:00100000:1.0:1789762849.280515:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074687232:0@lo:400:kworker.0 00000100:00100000:3.0:1789762849.280516:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074687616:0@lo:400:kworker.0 00000100:00100000:2.0:1789762849.280526:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762849.280535:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 452 00000100:00100000:0.0:1789762849.280553:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074686080:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.280571:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762849.280585:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074686976:0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.280593:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2759880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074687744:0@lo:400:kworker.0 00000100:00100000:3.0:1789762849.280602:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762849.280615:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074686080:12345-0@lo:400:kworker.0 Request processed in 68us (415us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762849.280623:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 452 00000100:00100000:0.0:1789762849.280657:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074687232 00000100:00100000:3.0:1789762849.280659:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074686080:0@lo:400:kworker.0 00000100:00100000:0.0:1789762849.280664:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074687232 00000100:00100000:1.0:1789762849.280665:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762849.280668:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762849.280711:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074686976 00000100:00100000:2.0:1789762849.280712:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074687488:0@lo:400:kworker.0 00000100:00100000:0.0:1789762849.280724:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074686976 00000100:00100000:3.0:1789762849.280738:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:1.0:1789762849.280748:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074685824 00000100:00100000:1.0:1789762849.280753:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074685824 00000100:00100000:3.0:1789762849.280761:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074686592:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762849.280762:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:0.0:1789762849.280773:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074687104:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.280784:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074686208 00000100:00100000:1.0:1789762849.280789:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074686208 00000100:00100000:2.0:1789762849.280798:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762849.280828:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074687104:12345-0@lo:400:kworker.0 Request processed in 54us (392us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762849.280828:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074686592:12345-0@lo:400:kworker.0 Request processed in 75us (506us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762849.280829:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074685696 00000100:00100000:1.0:1789762849.280834:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074685696 00000100:00100000:0.0:1789762849.280846:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:3.0:1789762849.280856:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00100000:1.0:1789762849.280887:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1789762849.280892:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:2.0:1789762849.280904:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:3.0:1789762849.280909:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:1.0:1789762849.280911:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074685824:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762849.280912:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074687232:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762849.280959:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074686208:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762849.280966:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074686976:12345-0@lo:400:kworker.0 00002000:00000024:1.0:1789762849.280979:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439418880 pending 0 free 3859808256 avail 3857711104 00002000:00000024:2.0:1789762849.280983:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439418880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762849.281051:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074687232:12345-0@lo:400:kworker.0 Request processed in 147us (478us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762849.281051:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074686976:12345-0@lo:400:kworker.0 Request processed in 84us (392us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762849.281059:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074685824:12345-0@lo:13:osp-pre-0-1.0 Request processed in 157us (810us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762849.281063:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074686208:12345-0@lo:13:osp-pre-0-0.0 Request processed in 109us (772us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762849.281067:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00100000:0.0:1789762849.281087:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:1.0:1789762849.281087:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:2.0:1789762849.281100:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00100000:1.0:1789762849.281127:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:0.0:1789762849.281139:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074685952 00000100:00100000:1.0:1789762849.281142:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074685696:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762849.281146:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074685952 00000100:00100000:3.0:1789762849.281148:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074687104:0@lo:400:kworker.0 00000100:00100000:2.0:1789762849.281153:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074687616 00000100:00100000:2.0:1789762849.281161:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074687616 00000100:00100000:3.0:1789762849.281168:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074686592:0@lo:400:kworker.0 00002000:00000024:1.0:1789762849.281170:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762849.281200:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:0.0:1789762849.281218:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1579850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074685952:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762849.281219:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074687744 00000100:00100000:2.0:1789762849.281224:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074687744 00000100:00100000:1.0:1789762849.281232:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074685696:12345-0@lo:13:osp-pre-1-1.0 Request processed in 90us (869us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762849.281240:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074686976:0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.281241:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:2.0:1789762849.281279:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074687488 00000100:00100000:2.0:1789762849.281283:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074687488 00000100:00100000:3.0:1789762849.281288:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074685824:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762849.281293:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1579850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074685952:12345-0@lo:400:kworker.0 Request processed in 81us (1105us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762849.281298:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074686720 00000100:00100000:1.0:1789762849.281304:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074686720 00000100:00100000:0.0:1789762849.281305:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00100000:2.0:1789762849.281322:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:3.0:1789762849.281328:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f58000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074685696:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762849.281343:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074687616:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.281345:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074686464 00000100:00100000:1.0:1789762849.281351:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074686464 00000100:00100000:0.0:1789762849.281364:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:0.0:1789762849.281379:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074686720:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.281396:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:3.0:1789762849.281396:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:1.0:1789762849.281409:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074686464:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762849.281417:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074687616:12345-0@lo:400:kworker.0 Request processed in 81us (824us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762849.281418:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074687744:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762849.281441:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074686720:12345-0@lo:400:kworker.0 Request processed in 62us (1006us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762849.281443:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00100000:0.0:1789762849.281461:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:2.0:1789762849.281492:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:1.0:1789762849.281499:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074686464:12345-0@lo:400:kworker.0 Request processed in 94us (988us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762849.281514:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074687488:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762849.281515:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762849.281515:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00000100:00100000:3.0:1789762849.281515:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074687744:12345-0@lo:400:kworker.0 Request processed in 98us (858us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762849.281541:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00080000:1.0:1789762849.281552:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762849.281556:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a09f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074686848:0@lo:400:kworker.0 00000100:00080000:1.0:1789762849.281558:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789762849.281570:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789762849.281579:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762849.281590:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074687488:12345-0@lo:400:kworker.0 Request processed in 86us (798us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762849.281599:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:3.0:1789762849.281604:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074687488:0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.281608:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074686336 00000100:00100000:1.0:1789762849.281615:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074686336 00000100:00100000:0.0:1789762849.281626:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762849.281634:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a09500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074686464:0@lo:400:kworker.0 00000100:00100000:2.0:1789762849.281659:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:0.0:1789762849.281669:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d275b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074687360:0@lo:400:kworker.0 00000100:00100000:2.0:1789762849.281670:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074686336:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762849.281681:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074686848 00000100:00100000:3.0:1789762849.281689:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074686848 00000100:00100000:1.0:1789762849.281695:0:3425:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967793 gen 2 00000004:00080000:1.0:1789762849.281702:0:3425:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37f7f5b480, transno 4294967793 00002000:00000024:2.0:1789762849.281705:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762849.281725:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c992b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074686208:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762849.281734:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762849.281758:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074686720:0@lo:400:kworker.0 00000100:00100000:3.0:1789762849.281764:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 457 00000100:00100000:2.0:1789762849.281771:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074686336:12345-0@lo:13:osp-pre-1-0.0 Request processed in 99us (254us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762849.281781:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074687232:0@lo:400:kworker.0 00000100:00100000:3.0:1789762849.281785:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074686848:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762849.281791:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074685952:0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.281813:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2759880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074687744:0@lo:400:kworker.0 00000100:00100000:2.0:1789762849.281819:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:0.0:1789762849.281871:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8a8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074686336:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762849.281872:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074686848:12345-0@lo:400:kworker.0 Request processed in 93us (245us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762849.281874:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762849.281884:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 457 00000100:00100000:2.0:1789762849.281895:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074687616:0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.281899:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074687360 00000100:00100000:0.0:1789762849.281902:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a09f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074686848:0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.281909:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074687360 00000100:00100000:1.0:1789762849.281983:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:1.0:1789762849.281992:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074687360:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.282038:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074687360:12345-0@lo:400:kworker.0 Request processed in 49us (309us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762849.282044:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00100000:0.0:1789762849.282075:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d275b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074687360:0@lo:400:kworker.0 00010000:00010000:1.0:1789762849.408062:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762850.432118:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:2.0:1789762850.826231:0:9000:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000020:00000080:2.0:1789762850.826239:0:9000:0:(genops.c:885:class_export_put()) final put ffff9c36cba8e000/fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000020:00000080:3.0:1789762850.826277:0:38:0:(genops.c:844:class_export_destroy()) destroying export ffff9c36cba8e000/fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 for lustre-OST0001 00000020:00000080:3.0:1789762850.826339:0:38:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9c36cba88800 for lustre-OST0001 00000100:00100000:0.0:1789762851.328135:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2759880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074687872:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762851.328198:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762851.328250:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074687872 00000100:00100000:2.0:1789762851.328273:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074687872 00000100:00100000:2.0:1789762851.328345:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 298 00000100:00100000:2.0:1789762851.328359:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074687872:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762851.328403:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762851.328500:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074687872:12345-0@lo:41:osp-pre-1-0.0 Request processed in 145us (301us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762851.328507:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 298 00000100:00100000:0.0:1789762851.328551:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2759880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074687872:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1789762851.456064:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762852.352165:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074688000:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762852.352244:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762852.352312:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074688000 00000100:00100000:0.0:1789762852.352324:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074688000 00000100:00100000:0.0:1789762852.352396:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 299 00000100:00100000:0.0:1789762852.352413:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074688000:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762852.352475:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762852.352557:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074688000:12345-0@lo:41:osp-pre-0-1.0 Request processed in 151us (314us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762852.352565:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 299 00000100:00100000:2.0:1789762852.352619:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074688000:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1789762852.480049:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762853.320054:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762853.320091:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762853.320115:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762853.320143:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762853.320158:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067411200 00000100:00100000:3.0:1789762853.320159:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067411328 00000100:00100000:3.0:1789762853.320173:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067411328 00000100:00100000:0.0:1789762853.320174:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067411200 00000100:00100000:2.0:1789762853.320184:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067411584 00000100:00100000:1.0:1789762853.320185:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762853.320199:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067411584 00000100:00100000:0.0:1789762853.320245:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 353 00000100:00100000:3.0:1789762853.320249:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067411456 00000100:00100000:3.0:1789762853.320253:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067411456 00000100:00100000:0.0:1789762853.320259:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1578050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067411200:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762853.320279:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 183 00000100:00100000:3.0:1789762853.320287:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067411840 00000100:00100000:3.0:1789762853.320292:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067411840 00000100:00100000:2.0:1789762853.320293:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067411584:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762853.320306:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762853.320329:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067411712 00000100:00100000:3.0:1789762853.320333:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067411712 00000100:00100000:0.0:1789762853.320351:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1578050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067411200:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 91us (304us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762853.320361:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 353 00000100:00100000:2.0:1789762853.320372:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067411584:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 81us (253us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762853.320383:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 183 00000100:00100000:3.0:1789762853.320401:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 458 00000100:00100000:3.0:1789762853.320415:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067411328:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:0.0:1789762853.320426:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762853.320432:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762853.320436:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762853.320472:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 459 00000100:00100000:1.0:1789762853.320487:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 460 00000100:00100000:0.0:1789762853.320491:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067411456:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762853.320500:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067411840:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762853.320509:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 461 00000100:00100000:3.0:1789762853.320528:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067411328:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 81us (397us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762853.320538:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067411456:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 54us (389us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762853.320548:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 458 00000100:00100000:1.0:1789762853.320550:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067411840:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 46us (360us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762853.320554:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067411712:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762853.320573:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 460 00000100:00100000:0.0:1789762853.320585:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 459 00000100:00100000:2.0:1789762853.320625:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067411712:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 84us (317us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762853.320635:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 461 00010000:00010000:1.0:1789762853.504082:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762854.400114:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074688640:0@lo:400:kworker.0 00000100:00100000:3.0:1789762854.400114:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ab9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074688256:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762854.400134:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074688768:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762854.400213:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762854.400219:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762854.400230:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c992b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074688896:0@lo:400:kworker.0 00000100:00100000:1.0:1789762854.400254:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074688768 00000100:00100000:1.0:1789762854.400269:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074688768 00000100:00100000:3.0:1789762854.400278:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762854.400305:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9254a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074689280:0@lo:400:kworker.0 00000100:00100000:2.0:1789762854.400306:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9255180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074689152:0@lo:400:kworker.0 00000100:00100000:3.0:1789762854.400314:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9257100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074689408:0@lo:400:kworker.0 00000100:00100000:1.0:1789762854.400336:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:0.0:1789762854.400351:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762854.400357:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074688768:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762854.400359:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762854.400370:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762854.400379:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9254e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074689792:0@lo:400:kworker.0 00000100:00100000:2.0:1789762854.400394:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9256a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074689664:0@lo:400:kworker.0 00000100:00100000:3.0:1789762854.400403:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074688128:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762854.400412:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:1.0:1789762854.400413:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439418880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762854.400452:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762854.400463:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762854.400488:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9255c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074690176:0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.400497:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074688256 00000100:00100000:3.0:1789762854.400497:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074688512:0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.400504:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074688256 00000100:00100000:1.0:1789762854.400513:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074688768:12345-0@lo:13:osp-pre-0-1.0 Request processed in 167us (301us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762854.400521:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00100000:2.0:1789762854.400526:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762854.400545:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074688128 00000100:00100000:1.0:1789762854.400552:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074688128 00000100:00100000:0.0:1789762854.400555:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:3.0:1789762854.400555:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762854.400557:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1789762854.400569:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9257480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074689024:0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.400570:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074688256:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762854.400580:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00002000:00000024:0.0:1789762854.400587:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439418880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762854.400593:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074688128:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762854.400606:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762854.400607:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8aa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074688384:0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789762854.400618:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762854.400636:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9254380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074689536:0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.400637:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074688256:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (267us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762854.400648:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:3.0:1789762854.400654:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762854.400666:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074688128:12345-0@lo:13:osp-pre-1-1.0 Request processed in 76us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762854.400673:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762854.400675:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:0.0:1789762854.400677:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074688896 00000100:00100000:0.0:1789762854.400682:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074688896 00000100:00100000:1.0:1789762854.400696:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074688384 00000100:00100000:1.0:1789762854.400701:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074688384 00000100:00100000:2.0:1789762854.400703:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9255880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074690048:0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.400716:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074689152 00000100:00100000:0.0:1789762854.400720:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074689152 00000100:00100000:2.0:1789762854.400729:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762854.400733:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 193 00000100:00100000:3.0:1789762854.400740:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ab9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074688256:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762854.400746:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074688384:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762854.400755:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 300 00002000:00000024:1.0:1789762854.400757:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762854.400770:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074688128:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762854.400772:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8ab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074688896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762854.400774:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074689792 00000100:00100000:2.0:1789762854.400780:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074689792 00000100:00100000:1.0:1789762854.400795:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074688384:12345-0@lo:13:osp-pre-1-0.0 Request processed in 54us (143us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762854.400801:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8aa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074688384:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762854.400807:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 193 00000100:00100000:2.0:1789762854.400815:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074690176 00000100:00100000:2.0:1789762854.400819:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074690176 00000100:00100000:0.0:1789762854.400834:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8ab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074688896:12345-0@lo:400:kworker.0 Request processed in 69us (556us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762854.400843:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 300 00000100:00100000:2.0:1789762854.400844:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074690048 00000100:00100000:2.0:1789762854.400848:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074690048 00000100:00100000:1.0:1789762854.400851:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:1.0:1789762854.400864:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074689792:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.400867:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 301 00000100:00100000:3.0:1789762854.400868:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074688512 00000100:00100000:3.0:1789762854.400875:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074688512 00000100:00100000:2.0:1789762854.400880:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:0.0:1789762854.400881:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074689152:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762854.400893:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074690176:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762854.400906:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074689792:12345-0@lo:400:kworker.0 Request processed in 47us (494us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762854.400912:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:3.0:1789762854.400950:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:1.0:1789762854.400955:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:3.0:1789762854.400965:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1578850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074688512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762854.400967:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074690048:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.400973:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074689152:12345-0@lo:400:kworker.0 Request processed in 94us (612us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762854.400977:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074690176:12345-0@lo:400:kworker.0 Request processed in 87us (450us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762854.400987:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 301 00000100:00100000:2.0:1789762854.400989:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00100000:1.0:1789762854.401025:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074690048:12345-0@lo:400:kworker.0 Request processed in 63us (297us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762854.401031:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762854.401037:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:2.0:1789762854.401038:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074688640 00000100:00080000:0.0:1789762854.401039:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762854.401039:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1578850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074688512:12345-0@lo:400:kworker.0 Request processed in 75us (485us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762854.401045:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074688640 00000100:00100000:3.0:1789762854.401057:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00100000:2.0:1789762854.401077:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074689280 00000100:00100000:2.0:1789762854.401080:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074689280 00000100:00100000:0.0:1789762854.401081:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074689664 00000100:00100000:1.0:1789762854.401084:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074689024 00000100:00100000:0.0:1789762854.401088:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074689664 00000100:00100000:1.0:1789762854.401089:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074689024 00000100:00100000:3.0:1789762854.401109:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074689536 00000100:00100000:2.0:1789762854.401114:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:3.0:1789762854.401116:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074689536 00000100:00100000:2.0:1789762854.401124:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074688640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762854.401137:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:0.0:1789762854.401144:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:1.0:1789762854.401151:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074689280:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.401157:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074689024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762854.401161:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074688640:12345-0@lo:400:kworker.0 Request processed in 37us (944us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762854.401166:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:2.0:1789762854.401174:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:3.0:1789762854.401181:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074689664:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762854.401191:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 466 00000100:00100000:2.0:1789762854.401203:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074689536:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762854.401211:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074689280:12345-0@lo:400:kworker.0 Request processed in 65us (859us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762854.401216:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074689024:12345-0@lo:400:kworker.0 Request processed in 62us (610us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762854.401223:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:0.0:1789762854.401229:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00100000:3.0:1789762854.401238:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074689664:12345-0@lo:400:kworker.0 Request processed in 61us (801us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762854.401249:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:2.0:1789762854.401257:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074689536:12345-0@lo:400:kworker.0 Request processed in 56us (583us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762854.401264:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9254a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074689280:0@lo:400:kworker.0 00000100:00100000:2.0:1789762854.401267:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 466 00000100:00100000:3.0:1789762854.401282:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762854.401285:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9254e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074689792:0@lo:400:kworker.0 00000100:00100000:2.0:1789762854.401299:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074689408 00000100:00100000:2.0:1789762854.401304:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074689408 00000100:00100000:1.0:1789762854.401307:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074688768:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1789762854.401317:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762854.401318:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9255500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074689920:0@lo:400:kworker.0 00000100:00080000:0.0:1789762854.401324:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762854.401340:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:2.0:1789762854.401350:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074689408:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.401356:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9257480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074689024:0@lo:400:kworker.0 00000100:00100000:3.0:1789762854.401362:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762854.401374:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9254380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074689536:0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.401390:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9255880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074690048:0@lo:400:kworker.0 00000100:00100000:1.0:1789762854.401398:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074689920 00000100:00100000:1.0:1789762854.401406:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074689920 00000100:00100000:2.0:1789762854.401412:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074689408:12345-0@lo:400:kworker.0 Request processed in 60us (129us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762854.401412:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c992b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074688896:0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.401413:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9255180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074689152:0@lo:400:kworker.0 00000100:00100000:2.0:1789762854.401429:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:3.0:1789762854.401455:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9257100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074689408:0@lo:400:kworker.0 00000100:00100000:1.0:1789762854.401456:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:0.0:1789762854.401460:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9256a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074689664:0@lo:400:kworker.0 00000100:00100000:1.0:1789762854.401469:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074689920:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.401487:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9255c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074690176:0@lo:400:kworker.0 00000100:00100000:2.0:1789762854.401494:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074688512:0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.401508:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074688640:0@lo:400:kworker.0 00000100:00100000:1.0:1789762854.401522:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074689920:12345-0@lo:400:kworker.0 Request processed in 54us (165us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762854.401529:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00100000:3.0:1789762854.401536:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9255500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074689920:0@lo:400:kworker.0 00010000:00010000:1.0:1789762854.528122:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762855.552059:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762856.448170:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9254e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074690304:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762856.448270:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762856.448343:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074690304 00000100:00100000:1.0:1789762856.448356:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074690304 00000100:00100000:1.0:1789762856.448440:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:1.0:1789762856.448454:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074690304:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762856.448513:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762856.448606:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074690304:12345-0@lo:41:osp-pre-1-0.0 Request processed in 156us (336us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762856.448616:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000100:00100000:0.0:1789762856.448665:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9254e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074690304:0@lo:41:osp-pre-1-0.0 00010000:00010000:2.0:1789762856.576039:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762857.472171:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8a8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074690432:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762857.472295:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762857.472339:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074690432 00000100:00100000:2.0:1789762857.472350:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074690432 00000100:00100000:2.0:1789762857.472429:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 303 00000100:00100000:2.0:1789762857.472440:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8abb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074690432:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762857.472488:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762857.472552:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8abb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074690432:12345-0@lo:41:osp-pre-0-1.0 Request processed in 113us (258us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762857.472561:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 303 00000100:00100000:3.0:1789762857.472779:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8a8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074690432:0@lo:41:osp-pre-0-1.0 00010000:00010000:1.0:1789762857.600081:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762858.440180:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762858.440220:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762858.440259:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762858.440263:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067411968 00000100:00100000:0.0:1789762858.440278:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067411968 00000100:00100000:2.0:1789762858.440280:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067412096 00000100:00100000:2.0:1789762858.440287:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067412096 00000100:00100000:1.0:1789762858.440292:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762858.440297:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067412352 00000100:00100000:3.0:1789762858.440304:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067412352 00000100:00100000:1.0:1789762858.440325:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762858.440346:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 355 00000100:00100000:3.0:1789762858.440358:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 188 00000100:00100000:2.0:1789762858.440361:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067412224 00000100:00100000:0.0:1789762858.440365:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1579450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067411968:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762858.440367:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067412224 00000100:00100000:1.0:1789762858.440371:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762858.440377:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067412352:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762858.440396:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067412480 00000100:00100000:1.0:1789762858.440401:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067412608 00000100:00100000:2.0:1789762858.440401:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067412480 00000100:00100000:1.0:1789762858.440404:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067412608 00000100:00100000:2.0:1789762858.440444:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 468 00000100:00100000:1.0:1789762858.440451:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 469 00000100:00100000:2.0:1789762858.440453:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9254a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067412096:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762858.440459:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9254700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067412224:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762858.440481:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067412352:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 107us (216us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762858.440482:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1579450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067411968:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 121us (309us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762858.440498:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 188 00000100:00100000:1.0:1789762858.440499:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9254700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067412224:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 36us (204us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762858.440499:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9254a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067412096:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 43us (273us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762858.440501:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 355 00000100:00100000:1.0:1789762858.440515:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 469 00000100:00100000:2.0:1789762858.440521:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 468 00000100:00100000:2.0:1789762858.440540:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 470 00000100:00100000:2.0:1789762858.440548:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9257800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067412480:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762858.440549:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 471 00000100:00100000:1.0:1789762858.440557:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9255f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067412608:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762858.440581:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9257800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067412480:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 33us (255us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762858.440590:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 470 00000100:00100000:1.0:1789762858.440591:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9255f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067412608:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 34us (222us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762858.440598:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 471 00000100:00080000:0.0:1789762858.440628:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762858.440635:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762858.440637:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789762858.624077:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762859.520162:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8a8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074690560:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762859.520284:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762859.520351:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9256300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074690688:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762859.520381:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762859.520437:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9257b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074691200:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.520468:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762859.520495:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9256a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074691712:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.520523:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762859.520548:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9257480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074692224:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.520573:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762859.520579:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074691200 00000100:00100000:0.0:1789762859.520592:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074691200 00000100:00100000:2.0:1789762859.520604:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:2.0:1789762859.520616:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8a8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074690816:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762859.520650:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074690560 00000100:00100000:2.0:1789762859.520657:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762859.520658:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074690560 00000100:00100000:0.0:1789762859.520661:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:0.0:1789762859.520674:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074691200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.520676:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9254000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074691072:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.520682:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9254e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074691328:0@lo:400:kworker.0 00000100:00100000:1.0:1789762859.520712:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074690688 00000100:00100000:2.0:1789762859.520717:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762859.520718:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074690688 00000100:00100000:1.0:1789762859.520752:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074690816 00000100:00100000:1.0:1789762859.520755:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074690816 00000100:00100000:2.0:1789762859.520765:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9255180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074691840:0@lo:400:kworker.0 00000100:00100000:1.0:1789762859.520785:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:0.0:1789762859.520789:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074691200:12345-0@lo:400:kworker.0 Request processed in 118us (321us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762859.520798:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074690560:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762859.520804:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00100000:2.0:1789762859.520805:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762859.520831:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9257100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074692352:0@lo:400:kworker.0 00002000:00000024:1.0:1789762859.520842:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762859.520856:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762859.520893:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:3.0:1789762859.520905:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074690688:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762859.520910:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9257b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074691200:0@lo:400:kworker.0 00000100:00100000:1.0:1789762859.520912:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074690560:12345-0@lo:13:osp-pre-1-0.0 Request processed in 117us (628us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762859.520950:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00002000:00000024:3.0:1789762859.520951:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762859.520962:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8a8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074690560:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762859.520975:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:1.0:1789762859.520988:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1aba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074690816:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762859.520995:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074690688:12345-0@lo:13:osp-pre-1-1.0 Request processed in 92us (613us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762859.521002:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00002000:00000024:1.0:1789762859.521015:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439418880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762859.521023:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9256300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074690688:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762859.521052:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074691712 00000100:00100000:3.0:1789762859.521058:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074691712 00000100:00100000:1.0:1789762859.521063:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1aba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074690816:12345-0@lo:13:osp-pre-0-0.0 Request processed in 76us (406us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762859.521063:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074691328 00000100:00100000:2.0:1789762859.521069:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074691328 00000100:00100000:1.0:1789762859.521073:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00100000:3.0:1789762859.521100:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074691840 00000100:00100000:3.0:1789762859.521105:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074691840 00000100:00080000:0.0:1789762859.521108:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762859.521114:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762859.521117:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00080000:0.0:1789762859.521120:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762859.521126:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762859.521130:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074691712:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.521144:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:0.0:1789762859.521152:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:3.0:1789762859.521156:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1aba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074691328:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762859.521167:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074691840:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.521175:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074691712:12345-0@lo:400:kworker.0 Request processed in 49us (651us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762859.521183:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00100000:3.0:1789762859.521208:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1aba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074691328:12345-0@lo:400:kworker.0 Request processed in 56us (491us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762859.521216:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074692224 00000100:00100000:3.0:1789762859.521217:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00100000:2.0:1789762859.521221:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074692224 00000100:00100000:0.0:1789762859.521224:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074691840:12345-0@lo:400:kworker.0 Request processed in 61us (424us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762859.521247:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00100000:2.0:1789762859.521257:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074692352 00000100:00100000:2.0:1789762859.521260:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074692352 00000100:00100000:3.0:1789762859.521270:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762859.521293:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:3.0:1789762859.521299:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9255c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074691584:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.521305:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074692224:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762859.521327:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074691072 00000100:00100000:3.0:1789762859.521334:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762859.521335:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074691072 00000100:00100000:1.0:1789762859.521340:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9256680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074690944:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.521347:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074692224:12345-0@lo:400:kworker.0 Request processed in 45us (774us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762859.521354:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:3.0:1789762859.521358:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9254380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074692096:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.521372:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:0.0:1789762859.521374:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074691584 00000100:00100000:0.0:1789762859.521380:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074691584 00000100:00100000:2.0:1789762859.521382:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074692352:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762859.521393:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762859.521416:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762859.521442:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 475 00000100:00100000:2.0:1789762859.521450:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074692352:12345-0@lo:400:kworker.0 Request processed in 69us (594us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762859.521456:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074691072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762859.521459:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9255500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074691456:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.521463:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00100000:3.0:1789762859.521464:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074692608:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762859.521482:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074692096 00000100:00100000:2.0:1789762859.521486:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074692096 00000100:00100000:1.0:1789762859.521510:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762859.521510:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762859.521515:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:0.0:1789762859.521518:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074691072:12345-0@lo:400:kworker.0 Request processed in 65us (247us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762859.521526:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074692096:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762859.521533:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 475 00000100:00100000:1.0:1789762859.521548:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9255880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074691968:0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.521553:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9254000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074691072:0@lo:400:kworker.0 00000100:00100000:0.0:1789762859.521559:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 476 00000100:00100000:2.0:1789762859.521567:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074692096:12345-0@lo:400:kworker.0 Request processed in 43us (169us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762859.521574:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074691584:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.521575:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9254380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074692096:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.521579:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:1.0:1789762859.521589:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762859.521616:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074692480:0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.521627:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074690944 00000100:00100000:3.0:1789762859.521632:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074690944 00000100:00100000:2.0:1789762859.521633:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074691968 00000100:00100000:0.0:1789762859.521638:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074691584:12345-0@lo:400:kworker.0 Request processed in 68us (302us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762859.521645:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762859.521645:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074691968 00000100:00100000:0.0:1789762859.521650:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 476 00000100:00100000:3.0:1789762859.521677:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:0.0:1789762859.521682:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 477 00000100:00100000:3.0:1789762859.521689:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d157a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074690944:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762859.521697:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074691968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.521701:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074692480 00000100:00100000:2.0:1789762859.521705:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074692480 00000100:00080000:1.0:1789762859.521714:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789762859.521721:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789762859.521726:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762859.521732:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:2.0:1789762859.521741:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074692480:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.521747:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d157a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074690944:12345-0@lo:400:kworker.0 Request processed in 61us (355us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762859.521754:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:0.0:1789762859.521764:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074691968:12345-0@lo:400:kworker.0 Request processed in 69us (173us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762859.521773:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 477 00000100:00100000:2.0:1789762859.521783:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074692480:12345-0@lo:400:kworker.0 Request processed in 43us (138us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762859.521788:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074691456 00000100:00100000:2.0:1789762859.521792:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:1.0:1789762859.521795:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074691456 00000100:00100000:3.0:1789762859.521809:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074692608 00000100:00100000:3.0:1789762859.521814:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074692608 00000100:00100000:0.0:1789762859.521827:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9255c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074691584:0@lo:400:kworker.0 00000100:00100000:1.0:1789762859.521835:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 305 00000100:00100000:2.0:1789762859.521846:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8a8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074690816:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762859.521850:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074691456:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.521854:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:2.0:1789762859.521862:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9254e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074691328:0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.521868:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074692608:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762859.521876:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9255180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074691840:0@lo:400:kworker.0 00002000:00000024:3.0:1789762859.521881:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439418880 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762859.521892:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9257100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074692352:0@lo:400:kworker.0 00000100:00100000:0.0:1789762859.521904:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9256680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074690944:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.521913:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9256a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074691712:0@lo:400:kworker.0 00000100:00100000:1.0:1789762859.521915:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074691456:12345-0@lo:400:kworker.0 Request processed in 70us (408us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762859.521948:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9257480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074692224:0@lo:400:kworker.0 00000100:00100000:0.0:1789762859.521952:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9255500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074691456:0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.521956:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074692608:12345-0@lo:13:osp-pre-0-1.0 Request processed in 89us (446us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762859.521966:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 305 00000100:00100000:3.0:1789762859.521970:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:0.0:1789762859.521981:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9255880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074691968:0@lo:400:kworker.0 00000100:00100000:0.0:1789762859.522008:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074692480:0@lo:400:kworker.0 00000100:00100000:1.0:1789762859.522020:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074692608:0@lo:13:osp-pre-0-1.0 00010000:00010000:0.0:1789762859.648109:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762860.673016:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762861.109155:0:16482:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.37@tcp 00000100:00100000:2.0:1789762861.109177:0:16482:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:2.0:1789762861.109185:0:16482:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:2.0:1789762861.109193:0:16482:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.37@tcp 00000100:00100000:2.0:1789762861.109201:0:16482:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:2.0:1789762861.109205:0:16482:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37ff91d400 rate 20 gen 0 00000100:00100000:2.0:1789762861.109209:0:16482:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:2.0:1789762861.109213:0:16482:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.37@tcp 00000100:00100000:2.0:1789762861.109221:0:16482:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:2.0:1789762861.109225:0:16482:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37ff91d200 rate 20 gen 0 00000100:00100000:1.0:1789762861.121336:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762861.121379:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067412736 00000100:00100000:3.0:1789762861.121440:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067412736 00000100:00100000:3.0:1789762861.121536:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 248 00000100:00080000:0.0:1789762861.121539:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762861.121553:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:14509:x1876702067412736:12345-192.168.203.37@tcp:101:mkdir.0 00010000:00010000:3.0:1789762861.121582:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762861.121607:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f45 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7ec expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.121705:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c800/0xdf5a7ac59f175f4c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.121723:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c800/0xdf5a7ac59f175f4c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.121740:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c800/0xdf5a7ac59f175f4c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762861.121796:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762861.121802:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762861.121829:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c800/0xdf5a7ac59f175f4c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.121870:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f175f45 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7ec expref: 7 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.121898:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c800/0xdf5a7ac59f175f4c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7ec expref: 6 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.121911:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762861.121916:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762861.121922:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d273c800, rc 0) 00000100:00100000:3.0:1789762861.122082:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:14509:x1876702067412736:12345-192.168.203.37@tcp:101:mkdir.0 Request processed in 530us (746us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762861.122092:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 248 00000100:00100000:3.0:1789762861.123876:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762861.124026:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762861.124086:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067412864 00000100:00100000:1.0:1789762861.124098:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067412864 00000100:00080000:3.0:1789762861.124140:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762861.124182:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067413120 00000100:00100000:1.0:1789762861.124189:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067413120 00000100:00100000:3.0:1789762861.124193:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 249 00000100:00100000:3.0:1789762861.124208:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067412864:12345-192.168.203.37@tcp:41:mkdir.0 00000100:00100000:1.0:1789762861.124241:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 250 00000100:00100000:1.0:1789762861.124257:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:14509:x1876702067413120:12345-192.168.203.37@tcp:36:mkdir.0 00000004:00000024:3.0:1789762861.124260:0:13389:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:3.0:1789762861.124331:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067412864:12345-192.168.203.37@tcp:41:mkdir.0 Request processed in 123us (454us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762861.124345:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 249 00000100:00100000:1.0:1789762861.124363:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074692736:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762861.124434:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762861.124507:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c9a9de40 going to sleep for 11 seconds 00000100:00100000:1.0:1789762861.124533:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074692736 00000100:00100000:1.0:1789762861.124536:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074692736 00000100:00100000:1.0:1789762861.124575:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 306 00000100:00100000:1.0:1789762861.124589:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074692736:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762861.124670:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074692736:12345-0@lo:1000:mdt00_004.0 Request processed in 88us (238us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.124676:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 306 00000100:00100000:0.0:1789762861.124736:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074692736:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.124833:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074692864:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.124881:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762861.124956:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074692864 00000100:00100000:1.0:1789762861.124963:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074692864 00000100:00100000:0.0:1789762861.124964:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f000 going to sleep for 11 seconds 00000100:00100000:1.0:1789762861.125010:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 307 00000100:00100000:1.0:1789762861.125019:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074692864:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762861.125278:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074692864:12345-0@lo:1000:mdt00_004.0 Request processed in 258us (393us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789762861.125288:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 307 00000100:00100000:0.0:1789762861.125302:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074692864:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1789762861.125357:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.125366:0:11692:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.125376:0:11692:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.125405:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074692992:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1789762861.125437:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762861.125493:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074692992 00000100:00100000:0.0:1789762861.125500:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f840 going to sleep for 11 seconds 00000100:00100000:3.0:1789762861.125501:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074692992 00000100:00100000:3.0:1789762861.125557:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 308 00000100:00100000:3.0:1789762861.125567:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074692992:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1789762861.125578:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762861.125590:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c600/0xdf5a7ac59f175f5a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f175f53 expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.125612:0:6424:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c800/0xdf5a7ac59f175f4c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7ec expref: 5 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.125627:0:6424:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c600/0xdf5a7ac59f175f5a lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f175f53 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.125669:0:6424:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c800/0xdf5a7ac59f175f4c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7ec expref: 5 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.125700:0:6424:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c800/0xdf5a7ac59f175f4c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7ec expref: 5 pid: 6418 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762861.125712:0:6424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a09f80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074693120:192.168.203.37@tcp:104: 00000100:00100000:3.0:1789762861.125800:0:6424:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1789762861.126826:0:6424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a09f80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074693120:192.168.203.37@tcp:104: 00010000:00010000:0.0:1789762861.126848:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c600/0xdf5a7ac59f175f5a lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f175f53 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.126858:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762861.126863:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1789762861.126867:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:0.0:1789762861.126878:0:6424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c600/0xdf5a7ac59f175f5a lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f175f53 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.126888:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d273c600, rc 0) 00000100:00100000:1.0:1789762861.126960:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067413248 00000100:00100000:1.0:1789762861.126967:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067413248 00010000:00010000:1.0:1789762861.126996:0:6404:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c800/0xdf5a7ac59f175f4c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7ec expref: 6 pid: 6418 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.127006:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:11692:x1876702074692992:12345-0@lo:101:mdt00_004.0 Request processed in 1430us (1556us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762861.127022:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 308 00000100:00100000:1.0:1789762861.127052:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 75 00000100:00100000:1.0:1789762861.127076:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067413248:12345-192.168.203.37@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1789762861.127081:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074692992:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1789762861.127103:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762861.127106:0:11692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.127118:0:6404:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c800/0xdf5a7ac59f175f4c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7ec expref: 6 pid: 6418 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.127125:0:11692:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.127141:0:6404:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c800/0xdf5a7ac59f175f4c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7ec expref: 6 pid: 6418 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.127164:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c800/0xdf5a7ac59f175f4c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7ec expref: 6 pid: 6418 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.127178:0:6404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762861.127184:0:6404:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c600/0xdf5a7ac59f175f5a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f175f53 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.127193:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c600/0xdf5a7ac59f175f5a lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f175f53 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.127200:0:6404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c600/0xdf5a7ac59f175f5a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f175f53 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.127225:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c600/0xdf5a7ac59f175f5a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f175f53 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762861.127234:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6404:1876702074693248:0@lo:105: 00000100:00100000:1.0:1789762861.127270:0:6404:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762861.127306:0:6449:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074693248 00000100:00100000:1.0:1789762861.127311:0:6404:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c9a9da80 going to sleep for 11 seconds 00000100:00100000:3.0:1789762861.127313:0:6449:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074693248 00000100:00100000:3.0:1789762861.127364:0:6449:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 49 00000100:00100000:3.0:1789762861.127374:0:6449:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6404:x1876702074693248:12345-0@lo:105: 00010000:00010000:3.0:1789762861.127403:0:6449:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.127419:0:6449:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.127432:0:6449:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.127461:0:6449:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.127473:0:6449:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.127487:0:6449:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.127520:0:6449:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c37d1768600) 00010000:00010000:0.0:1789762861.127540:0:11692:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.127552:0:11692:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762861.127572:0:11692:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1789762861.127583:0:6449:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6404:x1876702074693248:12345-0@lo:105: Request processed in 210us (311us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762861.127593:0:6449:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 49 00010000:00010000:0.0:1789762861.127606:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.127614:0:11692:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.127624:0:11692:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762861.127633:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6404:1876702074693248:0@lo:105: 00000100:00100000:0.0:1789762861.127645:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074693376:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1789762861.127653:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762861.127703:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762861.127711:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067413248:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 645us (842us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.127722:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 75 00000100:00100000:0.0:1789762861.127751:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f000 going to sleep for 11 seconds 00000100:00100000:3.0:1789762861.127832:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074693376 00000100:00100000:3.0:1789762861.127842:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074693376 00000100:00100000:1.0:1789762861.127900:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 309 00000100:00100000:1.0:1789762861.127908:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:11692:x1876702074693376:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1789762861.127943:0:6423:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762861.127963:0:6423:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f175f68 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f175f61 expref: -99 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.127987:0:6423:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f175f68 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f175f61 expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.128001:0:6423:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f175f68 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f175f61 expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.128009:0:6423:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743a00) 00010000:00010000:1.0:1789762861.128012:0:6423:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762861.128016:0:6423:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5932a00, rc 0) 00000100:00100000:1.0:1789762861.128054:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074693376:12345-0@lo:101:mdt00_004.0 Request processed in 147us (353us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.128068:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 309 00000100:00100000:0.0:1789762861.128096:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074693376:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1789762861.128116:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f175f68 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.128151:0:11692:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175f68 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762861.128172:0:11692:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1789762861.128204:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074693504:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.128238:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762861.128262:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fa80 going to sleep for 11 seconds 00000100:00100000:1.0:1789762861.128272:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074693504 00000100:00100000:1.0:1789762861.128278:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074693504 00000100:00100000:1.0:1789762861.128309:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 310 00000100:00100000:1.0:1789762861.128317:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074693504:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762861.128381:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074693504:12345-0@lo:1000:mdt00_004.0 Request processed in 62us (139us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.128401:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 310 00000100:00100000:0.0:1789762861.128415:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074693504:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.128455:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074693632:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.128490:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762861.128511:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fa80 going to sleep for 11 seconds 00000100:00100000:3.0:1789762861.128559:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074693632 00000100:00100000:3.0:1789762861.128568:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074693632 00000100:00100000:3.0:1789762861.128621:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 311 00000100:00100000:3.0:1789762861.128636:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074693632:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1789762861.129023:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074693632:12345-0@lo:1000:mdt00_004.0 Request processed in 385us (525us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789762861.129038:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 311 00000100:00100000:0.0:1789762861.129043:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074693632:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.129141:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074693760:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.129176:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762861.129208:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fa80 going to sleep for 11 seconds 00000100:00100000:3.0:1789762861.129258:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074693760 00000100:00100000:3.0:1789762861.129266:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074693760 00000100:00100000:3.0:1789762861.129316:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 312 00000100:00100000:3.0:1789762861.129332:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9a35850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074693760:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1789762861.129606:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9a35850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074693760:12345-0@lo:1000:mdt00_004.0 Request processed in 277us (426us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789762861.129620:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 312 00000100:00100000:0.0:1789762861.129632:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074693760:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.129703:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074693888:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.129763:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762861.129808:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1789762861.129811:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074693888 00000100:00100000:1.0:1789762861.129819:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074693888 00000100:00100000:1.0:1789762861.129870:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 313 00000100:00100000:1.0:1789762861.129887:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074693888:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762861.130007:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074693888:12345-0@lo:1000:mdt00_004.0 Request processed in 122us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.130021:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 313 00000100:00100000:0.0:1789762861.130053:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074693888:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.130155:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074694016:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.130211:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762861.130263:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074694016 00000100:00100000:0.0:1789762861.130265:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1789762861.130270:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074694016 00000100:00100000:1.0:1789762861.130332:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 314 00000100:00100000:1.0:1789762861.130359:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074694016:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762861.130436:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074694016:12345-0@lo:1000:mdt00_004.0 Request processed in 86us (229us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.130442:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 314 00000100:00100000:0.0:1789762861.130502:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074694016:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.130586:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074694144:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.130637:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762861.130671:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fd80 going to sleep for 11 seconds 00000100:00100000:1.0:1789762861.130680:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074694144 00000100:00100000:1.0:1789762861.130685:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074694144 00000100:00100000:1.0:1789762861.130714:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 315 00000100:00100000:1.0:1789762861.130723:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074694144:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762861.130779:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074694144:12345-0@lo:1000:mdt00_004.0 Request processed in 59us (144us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.130790:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 315 00000100:00100000:0.0:1789762861.130821:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074694144:0@lo:1000:mdt00_004.0 00000040:00080000:0.0:1789762861.133128:0:11692:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].9, 984 off39384 00000040:00080000:0.0:1789762861.133371:0:11692:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].9, 1136 off40464 00000004:00080000:0.0:1789762861.133379:0:11692:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37d176fa80 with version 20 00000004:00080000:1.0:1789762861.133437:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 20 rpc_version 20 00000020:00100000:1.0:1789762861.133462:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1789762861.133469:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37d8cf7480 x1876702074694400/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2416/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762861.133483:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf7480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074694400:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1789762861.133512:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762861.133535:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c9a9da80 going to sleep for 11 seconds 00000100:00100000:0.0:1789762861.133560:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074694400 00000100:00100000:0.0:1789762861.133567:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074694400 00000100:00100000:0.0:1789762861.133610:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:0.0:1789762861.133620:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7224:x1876702074694400:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762861.134679:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7224:x1876702074694400:12345-0@lo:1000:osp_up0-1.0 Request processed in 1059us (1163us total) trans 4294967405 rc 0/0 00000100:00100000:0.0:1789762861.134691:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00100000:1.0:1789762861.134780:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf7480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074694400:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1789762861.134839:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f175f6f lrc: 3/0,1 mode: --/PW res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.134855:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f175f6f lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.134871:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f175f6f lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762861.134888:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000218e128d reply state 00000000e707c855 transno 4294967401 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f175f6f lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.134919:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f175f6f lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.134976:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762861.134981:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762861.134989:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762861.134998:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x11:0x0]: returning size 16384 00000004:00080000:0.0:1789762861.135027:0:11692:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1789762861.135043:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.135059:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.135071:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.135097:0:11692:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1789762861.135102:0:11692:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1789762861.135108:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f175f68 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.135112:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762861.135120:0:11692:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f175f68 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.135127:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.135142:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768600 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:1.0:1789762861.135151:0:8069:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762861.135162:0:8069:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.135197:0:8069:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.135208:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14509:x1876702067413120:12345-192.168.203.37@tcp:36:mkdir.0 Request processed in 10951us (11178us total) trans 4294967401 rc 0/0 00010000:00010000:1.0:1789762861.135210:0:8069:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1789762861.135221:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 250 00010000:00010000:1.0:1789762861.135250:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.135262:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1768600/0xdf5a7ac59f175f53 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f175f5a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.135275:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762861.135278:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762861.135280:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762861.135296:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074694528:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1789762861.135349:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762861.135415:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074694528 00000100:00100000:0.0:1789762861.135422:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074694528 00000100:00100000:0.0:1789762861.135471:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 44 00000100:00100000:0.0:1789762861.135491:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3424:x1876702074694528:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762861.135504:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762861.135527:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c600/0xdf5a7ac59f175f5a lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f175f53 expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.135541:0:8070:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762861.135545:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762861.135603:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074694528:12345-0@lo:103:ldlm_bl.0 Request processed in 116us (259us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762861.135609:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 44 00000100:00100000:1.0:1789762861.135642:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074694528:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789762861.143799:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.143846:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067413376 00000100:00100000:0.0:1789762861.143855:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067413376 00000100:00100000:0.0:1789762861.143904:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 251 00000100:00100000:0.0:1789762861.143913:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a08000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:14510:x1876702067413376:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762861.143986:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762861.144000:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175f76 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7f3 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.144063:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f175f7d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.144080:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f175f7d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.144093:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f175f7d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762861.144131:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762861.144136:0:6420:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762861.144173:0:6420:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f175f7d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.144197:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175f76 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7f3 expref: 7 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.144213:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f175f7d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7f3 expref: 6 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.144221:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762861.144223:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762861.144226:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768400, rc 0) 00000100:00100000:0.0:1789762861.144271:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a08000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:14510:x1876702067413376:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 360us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762861.144278:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 251 00000100:00100000:3.0:1789762861.145578:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.145618:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067413632 00000100:00100000:0.0:1789762861.145623:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067413632 00000100:00100000:0.0:1789762861.145678:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 252 00000100:00100000:0.0:1789762861.145687:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:14510:x1876702067413632:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762861.145704:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762861.145706:0:13389:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1789762861.145728:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f175f7d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7f3 expref: 6 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.145747:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762861.145752:0:13389:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762861.145764:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f175f84 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c7fa expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.145873:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175f8b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.145882:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175f8b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.145891:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175f8b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.145901:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.145910:0:13389:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f175f68 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f175f61 expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.145964:0:13389:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.145995:0:13389:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f175f68 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xdf5a7ac59f175f61 expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.146010:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:13389:1876702074694656:0@lo:104: 00000100:00100000:0.0:1789762861.146051:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762861.146100:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f000 going to sleep for 11 seconds 00000100:00100000:3.0:1789762861.146115:0:6402:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074694656 00000100:00100000:3.0:1789762861.146123:0:6402:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074694656 00000100:00100000:3.0:1789762861.146226:0:6402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 50 00000100:00100000:3.0:1789762861.146249:0:6402:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:13389:x1876702074694656:12345-0@lo:104: 00010000:00010000:3.0:1789762861.146286:0:6402:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f175f68 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.146419:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:0.0:1789762861.146431:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:13389:1876702074694656:0@lo:104: 00010000:00010000:2.0:1789762861.146438:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f175f68 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.146443:0:6402:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to priority blp list, 0 blwis in pool 00010000:00010000:0.0:1789762861.146455:0:13389:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.146458:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769e00 already unused, calling callback (ffffffffc1f70ee0) 00000100:00100000:3.0:1789762861.146468:0:6402:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:13389:x1876702074694656:12345-0@lo:104: Request processed in 223us (406us total) trans 0 rc 0/0 00010000:00010000:2.0:1789762861.146472:0:6408:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xdf5a7ac59f175f68 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762861.146497:0:6408:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xdf5a7ac59f175f68 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762861.146497:0:6402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 50 00010000:00010000:2.0:1789762861.146546:0:6408:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f175f68 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.146551:0:6408:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762861.146589:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f175f68 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.146599:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769e00/0xdf5a7ac59f175f61 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f175f68 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762861.146619:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1abb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074694784:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762861.146624:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762861.146629:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762861.146633:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762861.146669:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762861.146718:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074694784 00000100:00100000:2.0:1789762861.146725:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074694784 00010000:00010000:2.0:1789762861.146751:0:6405:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f175f68 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xdf5a7ac59f175f61 expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762861.146786:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 76 00000100:00100000:2.0:1789762861.146795:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3425:x1876702074694784:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762861.146812:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:1.0:1789762861.166613:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967405 is committed 00010000:00010000:1.0:1789762861.169671:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f175f68 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xdf5a7ac59f175f61 expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.169685:0:6405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743a00) 00010000:00010000:1.0:1789762861.169689:0:6405:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.169697:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.169703:0:6405:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.169726:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762861.169744:0:13389:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.169774:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762861.169828:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074694784:12345-0@lo:103:ldlm_bl.0 Request processed in 23031us (23157us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.169844:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 76 00000100:00100000:3.0:1789762861.169858:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1abb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074694784:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762861.170166:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:13389:1876702074694912:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762861.170235:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762861.170267:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fb40 going to sleep for 11 seconds 00000100:00100000:3.0:1789762861.170290:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074694912 00000100:00100000:3.0:1789762861.170299:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074694912 00000100:00100000:3.0:1789762861.170348:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 317 00000100:00100000:3.0:1789762861.170358:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:13389:x1876702074694912:12345-0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762861.170468:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:13389:x1876702074694912:12345-0@lo:1000:mdt00_005.0 Request processed in 112us (233us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762861.170476:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 317 00000100:00100000:0.0:1789762861.170505:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-mdtlov_UUID:13389:1876702074694912:0@lo:1000:mdt00_005.0 00010000:00010000:0.0:1789762861.170580:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f175f99 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170594:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f175f99 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170603:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f175f99 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170617:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175f8b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170624:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175f8b lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170629:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175f8b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170634:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175f8b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170639:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762861.170644:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175f8b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170657:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175f8b lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170670:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762861.170673:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762861.170678:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175f8b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170683:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175f8b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170703:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170709:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170714:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170718:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762861.170720:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170729:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170734:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743a00) 00010000:00010000:0.0:1789762861.170736:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762861.170738:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170758:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175f92 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170811:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f175f84 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7fa expref: 7 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170829:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f175f99 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7fa expref: 6 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.170836:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a200) 00010000:00010000:0.0:1789762861.170839:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762861.170841:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768400, rc 0) 00000100:00100000:0.0:1789762861.170903:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:14510:x1876702067413632:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 25218us (25327us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762861.170910:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 252 00000100:00100000:3.0:1789762861.172510:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.172562:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067413888 00000100:00100000:0.0:1789762861.172568:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067413888 00000100:00100000:0.0:1789762861.172641:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 253 00000100:00100000:0.0:1789762861.172657:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14510:x1876702067413888:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762861.172672:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762861.172689:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175fa0 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c801 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762861.172786:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x11:0x0]: returning size 16384 00010000:00010000:0.0:1789762861.172809:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175fa0 lrc: 1/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c801 expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.172841:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762861.172844:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762861.172853:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175fa0 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c801 expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.172868:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769000, rc 0) 00000100:00100000:0.0:1789762861.172910:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14510:x1876702067413888:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 260us (406us total) trans 0 rc 301/301 00000100:00100000:0.0:1789762861.172988:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 253 00000100:00100000:3.0:1789762861.174120:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.174187:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067414016 00000100:00100000:0.0:1789762861.174194:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067414016 00000100:00100000:0.0:1789762861.174258:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 254 00000100:00100000:0.0:1789762861.174276:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:14510:x1876702067414016:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762861.174286:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762861.174300:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175fa7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c808 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.174366:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175fae lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.174376:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175fae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.174397:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175fae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762861.174436:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762861.174451:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762861.174486:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175fae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.174534:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f175fa7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c808 expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.174551:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175fae lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c808 expref: 7 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.174558:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762861.174561:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762861.174576:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768e00, rc 0) 00000100:00100000:0.0:1789762861.174618:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14510:x1876702067414016:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 351us (505us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762861.174626:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 254 00000100:00100000:1.0:1789762861.175743:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.175789:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067414272 00000100:00100000:2.0:1789762861.175798:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067414272 00000100:00100000:2.0:1789762861.175841:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 255 00000100:00100000:2.0:1789762861.175851:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14510:x1876702067414272:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762861.175871:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762861.175890:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7e00/0xdf5a7ac59f175fb5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c80f expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.175986:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f175fbc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.175997:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f175fbc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176008:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f175fbc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176024:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175fc3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176033:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175fc3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176041:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175fc3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176181:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f175fca lrc: 3/1,0 mode: --/PR res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176188:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f175fca lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176198:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f175fca lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176208:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f175fbc lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176212:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f175fbc lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176216:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f175fbc lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176220:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f175fbc lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176225:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762861.176228:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f175fbc lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176233:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f175fbc lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176243:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762861.176246:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762861.176249:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f175fbc lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176254:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f175fbc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176263:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175fc3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176270:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175fc3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176274:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175fc3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176277:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762861.176279:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175fc3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176283:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175fc3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176287:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3beee00) 00010000:00010000:2.0:1789762861.176289:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762861.176291:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175fc3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176294:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f175fc3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176317:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7e00/0xdf5a7ac59f175fb5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c80f expref: 9 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176325:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f175fca lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c80f expref: 8 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.176330:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a200) 00010000:00010000:2.0:1789762861.176331:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762861.176337:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc7800, rc 0) 00000100:00100000:2.0:1789762861.176382:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:14510:x1876702067414272:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 534us (641us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762861.176400:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 255 00000100:00100000:3.0:1789762861.177636:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762861.177671:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762861.177685:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067414656 00000100:00100000:1.0:1789762861.177692:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067414656 00000100:00100000:0.0:1789762861.177707:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067414528 00000100:00100000:0.0:1789762861.177714:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067414528 00000100:00100000:1.0:1789762861.177748:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 45 00000100:00100000:1.0:1789762861.177758:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a09180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2395:x1876702067414656:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762861.177771:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1789762861.177777:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 256 00010000:00010000:1.0:1789762861.177806:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f175fca lrc: 0/0,0 mode: --/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c80f expref: 8 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.177812:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14510:x1876702067414528:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762861.177828:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762861.177840:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f175fd1 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c816 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.177840:0:6406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a200) 00010000:00010000:1.0:1789762861.177856:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762861.177911:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a09180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067414656:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 153us (277us total) trans 0 rc 0/0 00000004:00200000:0.0:1789762861.177914:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x11:0x0]: returning size 16384 00000100:00100000:1.0:1789762861.177952:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 45 00010000:00010000:0.0:1789762861.178076:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f175fd1 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c816 expref: 9 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.178095:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762861.178097:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762861.178101:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f175fd1 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c816 expref: 9 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.178109:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768a00, rc 0) 00000100:00100000:0.0:1789762861.178138:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:14510:x1876702067414528:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 353us (468us total) trans 4294967402 rc 301/301 00000100:00100000:0.0:1789762861.178144:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 256 00000100:00100000:3.0:1789762861.179023:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.179066:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067414912 00000100:00100000:2.0:1789762861.179073:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067414912 00000100:00100000:2.0:1789762861.179107:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 257 00000100:00100000:2.0:1789762861.179113:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a08380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:14510:x1876702067414912:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:2.0:1789762861.179154:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7000/0xdf5a7ac59f175fdf lrc: 3/0,1 mode: --/PW res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.179163:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7000/0xdf5a7ac59f175fdf lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.179169:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7000/0xdf5a7ac59f175fdf lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762861.179307:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000bd11d13 reply state 00000000633b045e transno 4294967403 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7000/0xdf5a7ac59f175fdf lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.179316:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7000/0xdf5a7ac59f175fdf lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.179321:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:2.0:1789762861.179323:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:2.0:1789762861.179324:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762861.179330:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x11:0x0]: returning size 16384 00000100:00100000:2.0:1789762861.179378:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a08380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:14510:x1876702067414912:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 265us (356us total) trans 4294967403 rc 0/0 00000100:00100000:2.0:1789762861.179383:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 257 00000100:00100000:3.0:1789762861.180024:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.180068:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067415168 00000100:00100000:2.0:1789762861.180073:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067415168 00000100:00100000:2.0:1789762861.180111:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 69 00000100:00100000:2.0:1789762861.180118:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:14510:x1876702067415168:12345-192.168.203.37@tcp:35:lfs.0 00000100:00100000:2.0:1789762861.180221:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:14510:x1876702067415168:12345-192.168.203.37@tcp:35:lfs.0 Request processed in 104us (200us total) trans 4294967404 rc 0/0 00000100:00100000:2.0:1789762861.180226:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 69 00000100:00100000:3.0:1789762861.184557:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.184593:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067415296 00000100:00100000:0.0:1789762861.184599:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067415296 00000100:00100000:0.0:1789762861.184634:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 258 00000100:00100000:0.0:1789762861.184640:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:14511:x1876702067415296:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:0.0:1789762861.184654:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762861.184666:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175fe6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c81d expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.184706:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175fed lrc: 3/1,0 mode: --/PR res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.184719:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175fed lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.184731:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175fed lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762861.184757:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x11:0x0]: returning size 16384 00000004:00200000:0.0:1789762861.184770:0:13389:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762861.184791:0:13389:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175fed lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.184816:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769800/0xdf5a7ac59f175fe6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c81d expref: 13 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.184833:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175fed lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c81d expref: 12 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.184839:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762861.184842:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762861.184843:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768600, rc 0) 00000100:00100000:0.0:1789762861.184877:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:14511:x1876702067415296:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 237us (322us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762861.184882:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 258 00000100:00100000:3.0:1789762861.185839:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.185878:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067415552 00000100:00100000:0.0:1789762861.185882:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067415552 00000100:00100000:0.0:1789762861.185909:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 259 00000100:00100000:0.0:1789762861.185947:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:14511:x1876702067415552:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:0.0:1789762861.185976:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762861.185995:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175fed lrc: 0/0,0 mode: --/PR res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c81d expref: 12 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.186017:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762861.186021:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762861.186040:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175ff4 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.186066:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175ff4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.186077:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175ff4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762861.186294:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000090b023f8 reply state 0000000079e16a4e transno 4294967405 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175ff4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.186306:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175ff4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.186312:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762861.186315:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:0.0:1789762861.186317:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762861.186322:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x11:0x0]: returning size 16384 00000100:00100000:0.0:1789762861.186367:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:14511:x1876702067415552:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 451us (528us total) trans 4294967405 rc 0/0 00000100:00100000:0.0:1789762861.186373:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 259 00000100:00100000:3.0:1789762861.201260:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.201304:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067415680 00000100:00100000:0.0:1789762861.201313:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067415680 00000100:00100000:0.0:1789762861.201359:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 260 00000100:00100000:0.0:1789762861.201368:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14515:x1876702067415680:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:0.0:1789762861.201401:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762861.201418:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175ffb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c824 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201498:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176002 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201515:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176002 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201525:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176002 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201537:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176009 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201543:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176009 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201547:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176009 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201765:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176010 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201772:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176010 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201778:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176010 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762861.201795:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176010 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201808:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176002 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201813:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176002 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201826:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176002 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201831:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176002 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201835:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762861.201839:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176002 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201844:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176002 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201854:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762861.201856:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762861.201860:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176002 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201865:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176002 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201873:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176009 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201877:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176009 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201880:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176009 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201884:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762861.201890:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176009 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201895:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176009 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201900:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37e125a800) 00010000:00010000:0.0:1789762861.201901:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762861.201903:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176009 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201906:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176009 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201962:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f175ffb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c824 expref: 9 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201986:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176010 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c824 expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.201995:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a200) 00010000:00010000:0.0:1789762861.201997:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762861.201999:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769200, rc 0) 00000100:00100000:0.0:1789762861.202048:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:14515:x1876702067415680:12345-192.168.203.37@tcp:101:dd.500 Request processed in 682us (794us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762861.202053:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 260 00000100:00100000:3.0:1789762861.202912:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.202976:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067415808 00000100:00100000:2.0:1789762861.202988:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067415808 00000100:00100000:2.0:1789762861.203056:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 261 00000100:00100000:3.0:1789762861.203064:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.203076:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:14515:x1876702067415808:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:2.0:1789762861.203097:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762861.203115:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7e00/0xdf5a7ac59f176017 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c82b expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762861.203125:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067415936 00000100:00100000:1.0:1789762861.203131:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067415936 00010000:00010000:2.0:1789762861.203164:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f17601e lrc: 3/1,0 mode: --/PR res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762861.203172:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 46 00010000:00010000:2.0:1789762861.203175:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f17601e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762861.203183:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cba680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067415936:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762861.203189:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f17601e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.203194:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762861.203211:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176010 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c824 expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762861.203215:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x11:0x0]: returning size 16384 00000004:00200000:2.0:1789762861.203222:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00010000:00010000:1.0:1789762861.203229:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a200) 00010000:00010000:1.0:1789762861.203234:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762861.203248:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7e00/0xdf5a7ac59f176017 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c82b expref: 15 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.203278:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f17601e lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c82b expref: 14 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.203284:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:2.0:1789762861.203288:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00000100:00100000:1.0:1789762861.203290:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cba680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067415936:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 107us (227us total) trans 0 rc 0/0 00010000:00010000:2.0:1789762861.203291:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc7c00, rc 0) 00000100:00100000:1.0:1789762861.203304:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 46 00000100:00100000:2.0:1789762861.203338:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14515:x1876702067415808:12345-192.168.203.37@tcp:101:dd.500 Request processed in 266us (424us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762861.203348:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 261 00000100:00100000:3.0:1789762861.204157:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.204230:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067416192 00000100:00100000:0.0:1789762861.204237:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067416192 00000100:00100000:0.0:1789762861.204273:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 262 00000100:00100000:0.0:1789762861.204281:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14515:x1876702067416192:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:0.0:1789762861.204298:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762861.204308:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176025 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c832 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.204376:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17602c lrc: 3/0,1 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.204399:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17602c lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.204406:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17602c lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.204416:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176033 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x11:0x0].0x73550ecb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.204421:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176033 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.204426:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176033 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.204560:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074695040:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762861.204605:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762861.204641:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fb40 going to sleep for 11 seconds 00000100:00100000:1.0:1789762861.204662:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074695040 00000100:00100000:1.0:1789762861.204670:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074695040 00000100:00100000:1.0:1789762861.204704:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 318 00000100:00100000:1.0:1789762861.204711:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074695040:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762861.204821:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074695040:12345-0@lo:1000:mdt00_004.0 Request processed in 108us (212us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.204840:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 318 00000100:00100000:0.0:1789762861.204850:0:11692:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967405 gen 2 00000004:00080000:0.0:1789762861.204857:0:11692:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967405 committed transno 4294967405 00000100:00100000:0.0:1789762861.204884:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074695040:0@lo:1000:mdt00_004.0 00000004:00080000:0.0:1789762861.205211:0:11692:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0xa:0x0], index 0: 0 00000004:00080000:0.0:1789762861.205218:0:11692:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0xa:0x0], index 1: 0 00010000:00010000:0.0:1789762861.205357:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f17603a lrc: 3/1,0 mode: --/PR res: [0x240000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205364:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f17603a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205370:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f17603a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205412:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17602c lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205417:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17602c lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205421:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17602c lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205425:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17602c lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205429:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762861.205431:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17602c lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205435:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17602c lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205440:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762861.205442:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762861.205445:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17602c lrc: 3/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205458:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17602c lrc: 0/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762861.205466:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009eb4c29d reply state 00000000905f57cc transno 4294967406 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176033 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205476:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176033 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205481:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x73550ecb (ffff9c37e125ae00) 00010000:00010000:0.0:1789762861.205483:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762861.205484:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762861.205522:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176025 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c832 expref: 17 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205534:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f17603a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c832 expref: 16 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.205539:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x12:0x0].0x0 (ffff9c37e125ad00) 00010000:00010000:0.0:1789762861.205541:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762861.205544:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769c00, rc 0) 00000100:00100000:0.0:1789762861.205575:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:14515:x1876702067416192:12345-192.168.203.37@tcp:101:dd.500 Request processed in 1296us (1419us total) trans 4294967406 rc 0/0 00000100:00100000:0.0:1789762861.205580:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 262 00000100:00100000:3.0:1789762861.208145:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762861.208183:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067416320 00000100:00100000:1.0:1789762861.208190:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067416320 00000100:00080000:3.0:1789762861.208241:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762861.208247:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 157 00000100:00100000:1.0:1789762861.208261:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:14515:x1876702067416320:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:1.0:1789762861.208270:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762861.208281:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f176048 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c839 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762861.208299:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xa:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789762861.208310:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xa:0x0].0x0 since=334s v=(0 0) 00010000:00010000:1.0:1789762861.208314:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xa:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762861.208321:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f176048 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c839 expref: 6 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.208332:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f176048 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c839 expref: 6 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.208340:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5932a00, rc 0) 00000100:00100000:1.0:1789762861.208370:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:14515:x1876702067416320:12345-192.168.203.37@tcp:101:dd.500 Request processed in 116us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.208378:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 157 00000100:00100000:3.0:1789762861.220895:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.220947:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067416448 00000100:00100000:0.0:1789762861.220955:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067416448 00002000:00100000:0.0:1789762861.220998:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0xa:0x0] (0->1048575) req@ffff9c37ff685180 x1876702067416448/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.221022:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff685180 with x1876702067416448 ext(0->1048575) 00002000:00010000:0.0:1789762861.221028:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff685180 00000100:00100000:0.0:1789762861.221050:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269102247265 time 269102388121 state 0x0 ref 1 00000100:00100000:0.0:1789762861.221062:0:13301:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 6: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34100 gen 0 rate 20 depth 3 token 0 deadline 269102247265 time 269102388121 state 0x0 ref 1 00000100:00100000:0.0:1789762861.221093:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 3 deadline 334518167212 time 334468167212 state 0x1 ref 1 00000100:00100000:0.0:1789762861.221124:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 2 deadline 334518167212 time 334468201102 state 0x0 ref 1 00000100:00100000:0.0:1789762861.221133:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff685180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067416448:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762861.227348:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff685180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067416448:12345-192.168.203.37@tcp:4:dd.500 Request processed in 6208us (6450us total) trans 4294967794 rc 0/0 00000100:00100000:0.0:1789762861.227358:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 462 00000100:00100000:0.0:1789762861.227369:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 2 deadline 334518167212 time 334468201102 state 0x2 ref 0 00002000:00100000:0.0:1789762861.227381:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0xa:0x0] (0->1048575) req@ffff9c37ff685180 x1876702067416448/t4294967794(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.227402:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff685180 with x1876702067416448 ext(0->1048575) 00002000:00010000:0.0:1789762861.227406:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff685180 00000100:00100000:3.0:1789762861.228351:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.228415:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067416576 00000100:00100000:0.0:1789762861.228423:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067416576 00002000:00100000:0.0:1789762861.228474:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xa:0x0] (1048576->2097151) req@ffff9c37d8c8ed80 x1876702067416576/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.228482:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c8ed80 with x1876702067416576 ext(1048576->2097151) 00002000:00010000:0.0:1789762861.228486:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8c8ed80 00000100:00100000:0.0:1789762861.228496:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 2 deadline 334518167212 time 334468201102 state 0x0 ref 1 00000100:00100000:0.0:1789762861.228507:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 2 deadline 334518201102 time 334468201102 state 0x1 ref 1 00000100:00100000:0.0:1789762861.228530:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 1 deadline 334518201102 time 334475608615 state 0x0 ref 1 00000100:00100000:0.0:1789762861.228540:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067416576:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762861.231359:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2396:x1876702067416576:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2817us (3005us total) trans 4294967795 rc 0/0 00000100:00100000:0.0:1789762861.231366:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 463 00000100:00100000:0.0:1789762861.231373:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 1 deadline 334518201102 time 334475608615 state 0x2 ref 0 00002000:00100000:0.0:1789762861.231397:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xa:0x0] (1048576->2097151) req@ffff9c37d8c8ed80 x1876702067416576/t4294967795(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.231405:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8c8ed80 with x1876702067416576 ext(1048576->2097151) 00002000:00010000:0.0:1789762861.231409:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8c8ed80 00000100:00100000:3.0:1789762861.235618:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762861.235671:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067416704 00000100:00100000:1.0:1789762861.235684:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067416704 00002000:00100000:1.0:1789762861.235727:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0xa:0x0] (2097152->3145727) req@ffff9c37d184c700 x1876702067416704/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762861.235737:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184c700 with x1876702067416704 ext(2097152->3145727) 00002000:00010000:1.0:1789762861.235741:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184c700 00000100:00100000:1.0:1789762861.235759:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 1 deadline 334518201102 time 334475608615 state 0x0 ref 1 00000100:00100000:1.0:1789762861.235774:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 1 deadline 334525608615 time 334475608615 state 0x1 ref 1 00000100:00100000:1.0:1789762861.235807:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334525608615 time 334482884843 state 0x0 ref 1 00000100:00100000:1.0:1789762861.235818:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2395:x1876702067416704:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762861.238455:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:2395:x1876702067416704:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2634us (2836us total) trans 4294967796 rc 0/0 00000100:00100000:0.0:1789762861.238464:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 464 00000100:00100000:0.0:1789762861.238480:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334525608615 time 334482884843 state 0x2 ref 0 00002000:00100000:0.0:1789762861.238491:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0xa:0x0] (2097152->3145727) req@ffff9c37d184c700 x1876702067416704/t4294967796(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.238499:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184c700 with x1876702067416704 ext(2097152->3145727) 00002000:00010000:0.0:1789762861.238504:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184c700 00000100:00100000:1.0:1789762861.242895:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.242947:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067416960 00000100:00100000:1.0:1789762861.242948:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.242956:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067416960 00000100:00100000:2.0:1789762861.242989:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 158 00000100:00100000:2.0:1789762861.242998:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0+-99:2394:x1876702067416960:12345-192.168.203.37@tcp:8:dd.500 00000100:00100000:0.0:1789762861.243019:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067416832 00010000:00080000:2.0:1789762861.243028:0:9038:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp t0 exp 0000000000000000 cur 334 last 0 00000100:00100000:0.0:1789762861.243029:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067416832 00000020:00000080:2.0:1789762861.243070:0:9038:0:(genops.c:1385:class_connect()) connect: client fa6fc127-0d58-47ca-82d1-2c8ad7b086c1, cookie 0xdf5a7ac59f17604f 00002000:00100000:0.0:1789762861.243076:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xa:0x0] (3145728->4194303) req@ffff9c37d8cf4e00 x1876702067416832/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762861.243084:0:9038:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli fa6fc127-0d58-47ca-82d1-2c8ad7b086c1/ffff9c37d1024800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:0.0:1789762861.243086:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf4e00 with x1876702067416832 ext(3145728->4194303) 00002000:00010000:0.0:1789762861.243097:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf4e00 00000100:00100000:0.0:1789762861.243108:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334525608615 time 334482884843 state 0x0 ref 1 00002000:00100000:2.0:1789762861.243109:0:9038:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00000100:00100000:0.0:1789762861.243133:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334532884843 time 334482884843 state 0x1 ref 1 00002000:00080000:2.0:1789762861.243296:0:9038:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:2.0:1789762861.243333:0:9038:0:(import.c:65:import_set_state_nolock()) ffff9c37d1023800 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789762861.243404:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2394:x1876702067416960:12345-192.168.203.37@tcp:8:dd.500 Request processed in 405us (512us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762861.243414:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 158 00000100:00100000:1.0:1789762861.244150:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.244188:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067417088 00000100:00100000:2.0:1789762861.244193:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067417088 00000100:00100000:2.0:1789762861.244248:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 159 00000100:00080000:3.0:1789762861.244248:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762861.244263:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:14515:x1876702067417088:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:2.0:1789762861.244273:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762861.244289:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000012026acc/0xdf5a7ac59f176056 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c840 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762861.244313:0:9037:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0xa:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789762861.244330:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xa:0x0].0x0 since=334s v=(0 0) 00010000:00010000:2.0:1789762861.244336:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xa:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762861.244346:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000012026acc/0xdf5a7ac59f176056 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c840 expref: 7 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.244364:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000012026acc/0xdf5a7ac59f176056 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c840 expref: 7 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.244375:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc7e00, rc 0) 00000100:00100000:2.0:1789762861.244418:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14515:x1876702067417088:12345-192.168.203.37@tcp:101:dd.500 Request processed in 157us (268us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762861.244423:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 159 00000100:00100000:3.0:1789762861.252430:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.252486:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067417216 00000100:00100000:0.0:1789762861.252493:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067417216 00002000:00100000:0.0:1789762861.252522:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0xa:0x0] (0->1048575) req@ffff9c37d184f480 x1876702067417216/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.252531:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184f480 with x1876702067417216 ext(0->1048575) 00002000:00010000:0.0:1789762861.252534:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184f480 00000100:00100000:0.0:1789762861.252544:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334532884843 time 334482884843 state 0x1 ref 2 00000100:00100000:1.0:1789762861.259520:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.259576:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067417344 00000100:00100000:0.0:1789762861.259593:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067417344 00002000:00100000:0.0:1789762861.259616:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0xa:0x0] (1048576->2097151) req@ffff9c37d8cf5880 x1876702067417344/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.259625:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf5880 with x1876702067417344 ext(1048576->2097151) 00002000:00010000:0.0:1789762861.259628:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf5880 00000100:00100000:0.0:1789762861.259637:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334532884843 time 334482884843 state 0x1 ref 3 00000100:00100000:0.0:1789762861.285882:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334582949074 time 334532949074 state 0x1 ref 3 00000100:00100000:0.0:1789762861.285901:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:2396:x1876702067416832:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762861.289423:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:2396:x1876702067416832:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3525us (46472us total) trans 4294967797 rc 0/0 00000100:00100000:0.0:1789762861.289431:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 465 00002000:00100000:0.0:1789762861.289441:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0xa:0x0] (3145728->4194303) req@ffff9c37d8cf4e00 x1876702067416832/t4294967797(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.289449:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf4e00 with x1876702067416832 ext(3145728->4194303) 00002000:00010000:0.0:1789762861.289454:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf4e00 00000100:00100000:0.0:1789762861.335976:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334633036639 time 334583036639 state 0x1 ref 2 00000100:00100000:0.0:1789762861.336008:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067417216:12345-192.168.203.37@tcp:4:dd.500 00000001:00100000:2.0:1789762861.348373:0:9000:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000001:00080000:3.0:1789762861.359177:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967382 is committed 00000100:00100000:0.0:1789762861.359267:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067417216:12345-192.168.203.37@tcp:4:dd.500 Request processed in 23267us (106832us total) trans 4294967382 rc 0/0 00000100:00100000:0.0:1789762861.359280:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 466 00002000:00100000:0.0:1789762861.359301:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0xa:0x0] (0->1048575) req@ffff9c37d184f480 x1876702067417216/t4294967382(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.359310:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184f480 with x1876702067417216 ext(0->1048575) 00002000:00010000:0.0:1789762861.359318:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184f480 00000100:00100000:1.0:1789762861.361654:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762861.361713:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067417472 00000100:00100000:3.0:1789762861.361726:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067417472 00002000:00100000:3.0:1789762861.361769:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0xa:0x0] (2097152->3145727) req@ffff9c37d8cf5c00 x1876702067417472/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762861.361778:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf5c00 with x1876702067417472 ext(2097152->3145727) 00002000:00010000:3.0:1789762861.361781:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf5c00 00000100:00100000:3.0:1789762861.361799:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334633036639 time 334583036639 state 0x1 ref 2 00000100:00100000:3.0:1789762861.368722:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762861.368779:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067417600 00000100:00100000:1.0:1789762861.368784:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067417600 00002000:00100000:1.0:1789762861.368806:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0xa:0x0] (3145728->4194303) req@ffff9c37f6088700 x1876702067417600/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762861.368815:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f6088700 with x1876702067417600 ext(3145728->4194303) 00002000:00010000:1.0:1789762861.368819:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f6088700 00000100:00100000:1.0:1789762861.368829:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334633036639 time 334583036639 state 0x1 ref 3 00000100:00100000:1.0:1789762861.375155:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.375211:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067417728 00000100:00100000:0.0:1789762861.375216:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067417728 00002000:00100000:0.0:1789762861.375240:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0xa:0x0] (4194304->5242879) req@ffff9c37d8cf6d80 x1876702067417728/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.375248:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf6d80 with x1876702067417728 ext(4194304->5242879) 00002000:00010000:0.0:1789762861.375252:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf6d80 00000100:00100000:0.0:1789762861.375261:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334633036639 time 334583036639 state 0x1 ref 4 00000100:00100000:1.0:1789762861.381693:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.381742:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067417856 00000100:00100000:0.0:1789762861.381749:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067417856 00002000:00100000:0.0:1789762861.381776:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xa:0x0] (5242880->6291455) req@ffff9c37d8cf7480 x1876702067417856/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.381788:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf7480 with x1876702067417856 ext(5242880->6291455) 00002000:00010000:0.0:1789762861.381793:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf7480 00000100:00100000:0.0:1789762861.381805:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334633036639 time 334583036639 state 0x1 ref 5 00000100:00100000:0.0:1789762861.386016:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334683089175 time 334633089175 state 0x1 ref 5 00000100:00100000:0.0:1789762861.386038:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067417344:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762861.389213:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.389235:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2398:x1876702067417344:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3205us (129720us total) trans 4294967383 rc 0/0 00000100:00100000:0.0:1789762861.389252:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 467 00002000:00100000:0.0:1789762861.389260:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0xa:0x0] (1048576->2097151) req@ffff9c37d8cf5880 x1876702067417344/t4294967383(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.389269:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf5880 with x1876702067417344 ext(1048576->2097151) 00002000:00010000:0.0:1789762861.389273:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf5880 00000100:00100000:2.0:1789762861.389275:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067417984 00000100:00100000:2.0:1789762861.389282:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067417984 00002000:00100000:2.0:1789762861.389308:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0xa:0x0] (6291456->7340031) req@ffff9c37d8cf5f80 x1876702067417984/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762861.389317:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf5f80 with x1876702067417984 ext(6291456->7340031) 00002000:00010000:2.0:1789762861.389321:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf5f80 00000100:00100000:2.0:1789762861.389339:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334683089175 time 334633089175 state 0x1 ref 5 00000100:00100000:1.0:1789762861.395836:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.395882:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067418112 00000100:00100000:0.0:1789762861.395888:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067418112 00002000:00100000:0.0:1789762861.395908:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0xa:0x0] (7340032->8388607) req@ffff9c37d8cf7b80 x1876702067418112/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.395930:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf7b80 with x1876702067418112 ext(7340032->8388607) 00002000:00010000:0.0:1789762861.395933:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf7b80 00000100:00100000:0.0:1789762861.395943:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334683089175 time 334633089175 state 0x1 ref 6 00000100:00100000:1.0:1789762861.403403:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.403461:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067418240 00000100:00100000:0.0:1789762861.403470:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067418240 00002000:00100000:0.0:1789762861.403511:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0xa:0x0] (4194304->5242879) req@ffff9c37d8cf4000 x1876702067418240/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.403520:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf4000 with x1876702067418240 ext(4194304->5242879) 00002000:00010000:0.0:1789762861.403524:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf4000 00000100:00100000:0.0:1789762861.403540:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334683089175 time 334633089175 state 0x1 ref 7 00000100:00100000:3.0:1789762861.409937:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.409981:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067418368 00000100:00100000:0.0:1789762861.409987:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067418368 00002000:00100000:0.0:1789762861.410008:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0xa:0x0] (5242880->6291455) req@ffff9c37f608b100 x1876702067418368/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.410026:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f608b100 with x1876702067418368 ext(5242880->6291455) 00002000:00010000:0.0:1789762861.410030:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f608b100 00000100:00100000:0.0:1789762861.410040:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334683089175 time 334633089175 state 0x1 ref 8 00000100:00100000:0.0:1789762861.436078:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334733145153 time 334683145153 state 0x1 ref 8 00000100:00100000:0.0:1789762861.436100:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:2398:x1876702067417472:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762861.439708:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:2398:x1876702067417472:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3611us (78050us total) trans 4294967384 rc 0/0 00000100:00100000:0.0:1789762861.439721:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 468 00002000:00100000:0.0:1789762861.439734:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0xa:0x0] (2097152->3145727) req@ffff9c37d8cf5c00 x1876702067417472/t4294967384(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.439747:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf5c00 with x1876702067417472 ext(2097152->3145727) 00002000:00010000:0.0:1789762861.439755:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf5c00 00000100:00100000:3.0:1789762861.441745:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.441792:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067418496 00000100:00100000:0.0:1789762861.441801:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067418496 00002000:00100000:0.0:1789762861.441838:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0xa:0x0] (6291456->7340031) req@ffff9c37f6088a80 x1876702067418496/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.441849:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f6088a80 with x1876702067418496 ext(6291456->7340031) 00002000:00010000:0.0:1789762861.441854:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f6088a80 00000100:00100000:0.0:1789762861.441870:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334733145153 time 334683145153 state 0x1 ref 8 00000100:00100000:1.0:1789762861.448447:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.448495:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067418624 00000100:00100000:0.0:1789762861.448502:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067418624 00002000:00100000:0.0:1789762861.448527:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0xa:0x0] (7340032->8388607) req@ffff9c37d8cf4700 x1876702067418624/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.448537:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf4700 with x1876702067418624 ext(7340032->8388607) 00002000:00010000:0.0:1789762861.448541:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf4700 00000100:00100000:0.0:1789762861.448552:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334733145153 time 334683145153 state 0x1 ref 9 00000100:00100000:1.0:1789762861.448570:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.448637:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067418880 00000100:00100000:2.0:1789762861.448643:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067418880 00000100:00100000:2.0:1789762861.448680:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 70 00000100:00100000:2.0:1789762861.448690:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbd050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:14515:x1876702067418880:12345-192.168.203.37@tcp:35:dd.500 00000100:00100000:2.0:1789762861.448989:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbd050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:14515:x1876702067418880:12345-192.168.203.37@tcp:35:dd.500 Request processed in 301us (419us total) trans 4294967407 rc 0/0 00000100:00100000:2.0:1789762861.448995:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 70 00000100:00100000:1.0:1789762861.483109:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.483173:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067419136 00000100:00100000:2.0:1789762861.483185:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067419136 00000100:00100000:2.0:1789762861.483254:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 263 00000100:00100000:2.0:1789762861.483266:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:14526:x1876702067419136:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762861.483294:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762861.483324:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f17605d lrc: 2/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c847 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.483472:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6800/0xdf5a7ac59f176064 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.483487:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6800/0xdf5a7ac59f176064 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.483498:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6800/0xdf5a7ac59f176064 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.483510:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f17606b lrc: 3/0,1 mode: --/PW res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.483534:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f17606b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.483539:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f17606b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762861.483927:0:6420:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0xb:0x0], index 1: 0 00000004:00080000:2.0:1789762861.483965:0:6420:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0xb:0x0], index 0: 0 00010000:00010000:2.0:1789762861.484193:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6e00/0xdf5a7ac59f176072 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484203:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6e00/0xdf5a7ac59f176072 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484212:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6e00/0xdf5a7ac59f176072 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484251:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6800/0xdf5a7ac59f176064 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484257:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6800/0xdf5a7ac59f176064 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484262:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6800/0xdf5a7ac59f176064 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484267:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6800/0xdf5a7ac59f176064 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484271:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762861.484276:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6800/0xdf5a7ac59f176064 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484289:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6800/0xdf5a7ac59f176064 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484301:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:2.0:1789762861.484303:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762861.484307:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6800/0xdf5a7ac59f176064 lrc: 3/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484312:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6800/0xdf5a7ac59f176064 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762861.484325:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000475fb9f8 reply state 000000000ac0d7ab transno 4294967408 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f17606b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484333:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f17606b lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484338:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x2b0e463d (ffff9c37d3bee100) 00010000:00010000:2.0:1789762861.484340:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762861.484341:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762861.484366:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f17605d lrc: 0/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c847 expref: 21 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484376:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6e00/0xdf5a7ac59f176072 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c847 expref: 20 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.484396:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x13:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762861.484398:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762861.484400:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc6e00, rc 0) 00000100:00100000:2.0:1789762861.484464:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:14526:x1876702067419136:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1201us (1366us total) trans 4294967408 rc 0/0 00000100:00100000:2.0:1789762861.484470:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 263 00000100:00100000:0.0:1789762861.486117:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334783186720 time 334733186720 state 0x1 ref 9 00000100:00100000:0.0:1789762861.486134:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:2398:x1876702067417600:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762861.488971:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+16:2398:x1876702067417600:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2839us (120286us total) trans 4294967385 rc 0/0 00000100:00100000:0.0:1789762861.488986:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 469 00002000:00100000:0.0:1789762861.488997:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0xa:0x0] (3145728->4194303) req@ffff9c37f6088700 x1876702067417600/t4294967385(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.489008:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f6088700 with x1876702067417600 ext(3145728->4194303) 00002000:00010000:0.0:1789762861.489014:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f6088700 00000100:00100000:1.0:1789762861.489171:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.489214:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067419264 00000100:00100000:0.0:1789762861.489219:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067419264 00000100:00100000:0.0:1789762861.489266:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 160 00000100:00100000:0.0:1789762861.489274:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+16:14526:x1876702067419264:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762861.489283:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762861.489328:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c84e expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762861.489349:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0xb:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789762861.489366:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=334s v=(0 0) 00010000:00010000:0.0:1789762861.489371:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762861.489398:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 17 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.489417:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 17 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.489445:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769400, rc 0) 00000100:00100000:3.0:1789762861.489490:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.489507:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:14526:x1876702067419264:12345-192.168.203.37@tcp:101:dd.0 Request processed in 232us (335us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762861.489514:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 160 00000100:00100000:2.0:1789762861.489549:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067419392 00000100:00100000:2.0:1789762861.489556:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067419392 00000100:00100000:2.0:1789762861.489597:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 161 00000100:00100000:2.0:1789762861.489606:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:14527:x1876702067419392:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762861.489615:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762861.489625:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001c40f06e/0xdf5a7ac59f176087 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c855 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.489660:0:9038:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 18 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.489667:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762861.489670:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789762861.489673:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001c40f06e/0xdf5a7ac59f176087 lrc: 4/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c855 expref: 18 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.489717:0:9038:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 18 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.489735:0:9038:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 18 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762861.489750:0:9038:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1abb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:9038:1876702074695168:192.168.203.37@tcp:104: 00000100:00100000:2.0:1789762861.489827:0:9038:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37eebc6600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762861.493672:0:9038:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1abb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:9038:1876702074695168:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762861.493699:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001c40f06e/0xdf5a7ac59f176087 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c855 expref: 18 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.493712:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc7a00, rc 0) 00000100:00100000:2.0:1789762861.493745:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:14527:x1876702067419392:12345-192.168.203.37@tcp:101:dd.0 Request processed in 4142us (4259us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762861.493751:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 161 00000100:00100000:1.0:1789762861.499875:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.499933:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067419648 00000100:00100000:2.0:1789762861.499939:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067419648 00000100:00100000:2.0:1789762861.499976:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 162 00000100:00100000:2.0:1789762861.499985:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:14530:x1876702067419648:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762861.499997:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762861.500010:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f17608e lrc: 2/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c863 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762861.500026:0:9037:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xb:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789762861.500036:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=334s v=(0 0) 00010000:00010000:2.0:1789762861.500039:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762861.500044:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f17608e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c863 expref: 19 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.500071:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f17608e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c863 expref: 19 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.500089:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc6600, rc 0) 00000100:00100000:2.0:1789762861.500117:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:14530:x1876702067419648:12345-192.168.203.37@tcp:101:dd.0 Request processed in 134us (244us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762861.500122:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 162 00000100:00100000:0.0:1789762861.503176:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762861.503220:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067419776 00000100:00100000:1.0:1789762861.503226:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067419776 00000100:00100000:1.0:1789762861.503256:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 163 00000100:00100000:1.0:1789762861.503263:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:14529:x1876702067419776:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762861.503271:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762861.503279:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f176095 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c86a expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.503293:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789762861.503296:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(2 0) 00010000:00010000:1.0:1789762861.503301:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f176095 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c86a expref: 19 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.503309:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f176095 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c86a expref: 19 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.503317:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5932400, rc 0) 00000100:00100000:1.0:1789762861.503344:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:14529:x1876702067419776:12345-192.168.203.37@tcp:101:dd.0 Request processed in 82us (171us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.503350:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 163 00000100:00100000:3.0:1789762861.505366:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.505477:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067419904 00000100:00100000:2.0:1789762861.505483:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067419904 00002000:00100000:2.0:1789762861.505508:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0xb:0x0] (0->1048575) req@ffff9c37f608a300 x1876702067419904/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762861.505516:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f608a300 with x1876702067419904 ext(0->1048575) 00010000:00010000:2.0:1789762861.505522:0:14170:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 19 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.505533:0:14170:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 19 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762861.505540:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9c37f608a300 00000100:00100000:2.0:1789762861.505553:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126500 gen 0 rate 20 depth 3 token 2 deadline 298738753072 time 298745823862 state 0x0 ref 1 00000100:00100000:2.0:1789762861.505560:0:14170:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 6: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126500 gen 0 rate 20 depth 3 token 2 deadline 298738753072 time 298745823862 state 0x0 ref 1 00000100:00100000:2.0:1789762861.505590:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 3 deadline 334752766028 time 334752666028 state 0x1 ref 1 00000100:00100000:2.0:1789762861.505611:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 334752766028 time 334752688962 state 0x0 ref 1 00000100:00100000:2.0:1789762861.505627:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:2395:x1876702067419904:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762861.510592:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:2395:x1876702067419904:12345-192.168.203.37@tcp:4:dd.0 Request processed in 4965us (5225us total) trans 4294967386 rc 0/0 00000100:00100000:2.0:1789762861.510599:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 4 00000100:00100000:2.0:1789762861.510605:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 334752766028 time 334752688962 state 0x2 ref 0 00002000:00100000:2.0:1789762861.510613:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0xb:0x0] (0->1048575) req@ffff9c37f608a300 x1876702067419904/t4294967386(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762861.510620:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f608a300 with x1876702067419904 ext(0->1048575) 00010000:00010000:2.0:1789762861.510625:0:14170:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 21 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.510636:0:14170:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 21 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762861.510642:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9c37f608a300 00000100:00100000:3.0:1789762861.516853:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.516937:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067420032 00000100:00100000:2.0:1789762861.516943:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067420032 00000100:00100000:2.0:1789762861.516971:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 164 00000100:00100000:2.0:1789762861.516978:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:14535:x1876702067420032:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762861.516987:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762861.516996:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17609c lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c871 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.517023:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(2 0) 00010000:00010000:2.0:1789762861.517026:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762861.517031:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17609c lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 5 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c871 expref: 22 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.517040:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17609c lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 5 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c871 expref: 22 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.517048:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc6800, rc 0) 00000100:00100000:2.0:1789762861.517075:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:14535:x1876702067420032:12345-192.168.203.37@tcp:101:dd.0 Request processed in 98us (224us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762861.517080:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 164 00000100:00100000:0.0:1789762861.518631:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.518705:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067420160 00000100:00100000:2.0:1789762861.518711:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067420160 00000100:00100000:3.0:1789762861.518718:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.518765:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067419520 00000100:00100000:2.0:1789762861.518768:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067419520 00000100:00100000:0.0:1789762861.518777:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 165 00000100:00100000:0.0:1789762861.518787:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:14534:x1876702067420160:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762861.518795:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1789762861.518799:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 166 00010000:00010000:0.0:1789762861.518805:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000020398f3e/0xdf5a7ac59f1760a3 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c878 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762861.518813:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:14536:x1876702067419520:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762861.518825:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762861.518827:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(3 0) 00010000:00010000:0.0:1789762861.518832:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762861.518834:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000081703a57/0xdf5a7ac59f1760aa lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c85c expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.518838:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000020398f3e/0xdf5a7ac59f1760a3 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c878 expref: 24 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.518855:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762861.518860:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(5 0) 00010000:00010000:2.0:1789762861.518864:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000081703a57/0xdf5a7ac59f1760aa lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 7 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c85c expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.518887:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000081703a57/0xdf5a7ac59f1760aa lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c85c expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.518888:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000020398f3e/0xdf5a7ac59f1760a3 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c878 expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.518906:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc7600, rc 0) 00010000:00010000:0.0:1789762861.518913:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769800, rc 0) 00000100:00100000:2.0:1789762861.518962:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:14536:x1876702067419520:12345-192.168.203.37@tcp:101:dd.0 Request processed in 154us (246us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762861.518968:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 166 00000100:00100000:0.0:1789762861.518979:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:14534:x1876702067420160:12345-192.168.203.37@tcp:101:dd.0 Request processed in 193us (347us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762861.518986:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 165 00000100:00100000:0.0:1789762861.524422:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.524471:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067420288 00000100:00100000:2.0:1789762861.524477:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067420288 00000100:00100000:2.0:1789762861.524503:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 167 00000100:00100000:2.0:1789762861.524509:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:14537:x1876702067420288:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762861.524517:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762861.524525:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000071d3bc94/0xdf5a7ac59f1760b1 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c87f expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.524542:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(5 0) 00010000:00010000:2.0:1789762861.524545:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:2.0:1789762861.524549:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000071d3bc94/0xdf5a7ac59f1760b1 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c87f expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.524557:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000071d3bc94/0xdf5a7ac59f1760b1 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c87f expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.524565:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc6200, rc 0) 00000100:00100000:2.0:1789762861.524591:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:14537:x1876702067420288:12345-192.168.203.37@tcp:101:dd.0 Request processed in 82us (170us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762861.524603:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 167 00000100:00100000:1.0:1789762861.532261:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.532307:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067420416 00000100:00100000:0.0:1789762861.532312:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067420416 00002000:00100000:0.0:1789762861.532336:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0xb:0x0] (0->1048575) req@ffff9c37d8ccc380 x1876702067420416/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762861.532345:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccc380 with x1876702067420416 ext(0->1048575) 00002000:00010000:0.0:1789762861.532348:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8ccc380 00000100:00100000:0.0:1789762861.532360:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305146227975 time 305146319483 state 0x0 ref 1 00000100:00100000:0.0:1789762861.532366:0:8057:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 6: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126b00 gen 0 rate 20 depth 3 token 0 deadline 305146227975 time 305146319483 state 0x0 ref 1 00000100:00100000:0.0:1789762861.532408:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 3 deadline 334779579150 time 334779479150 state 0x1 ref 1 00000100:00100000:0.0:1789762861.532534:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334779579150 time 334779610938 state 0x0 ref 1 00000100:00100000:0.0:1789762861.532543:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:2397:x1876702067420416:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762861.537194:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:2397:x1876702067420416:12345-192.168.203.37@tcp:4:dd.0 Request processed in 4648us (4931us total) trans 4294967798 rc 0/0 00000100:00100000:0.0:1789762861.537202:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 478 00000100:00100000:0.0:1789762861.537209:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334779579150 time 334779610938 state 0x2 ref 0 00002000:00100000:0.0:1789762861.537219:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xb:0x0] (0->1048575) req@ffff9c37d8ccc380 x1876702067420416/t4294967798(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762861.537226:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccc380 with x1876702067420416 ext(0->1048575) 00002000:00010000:0.0:1789762861.537230:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8ccc380 00000100:00100000:0.0:1789762861.537249:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334834326842 time 334784326842 state 0x1 ref 8 00000100:00100000:0.0:1789762861.537260:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:2398:x1876702067417728:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762861.537613:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762861.537649:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067420544 00000100:00100000:3.0:1789762861.537654:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067420544 00002000:00100000:3.0:1789762861.537676:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0xb:0x0] (2097152->3145727) req@ffff9c37d8ccdc00 x1876702067420544/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762861.537684:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccdc00 with x1876702067420544 ext(2097152->3145727) 00010000:00010000:3.0:1789762861.537689:0:8058:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 25 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.537701:0:8058:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 25 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762861.537707:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9c37d8ccdc00 00000100:00100000:3.0:1789762861.537717:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 334752766028 time 334752688962 state 0x0 ref 1 00000100:00100000:3.0:1789762861.537726:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 334752788962 time 334752688962 state 0x1 ref 1 00000100:00100000:3.0:1789762861.537748:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 334752788962 time 334784826762 state 0x0 ref 1 00000100:00100000:3.0:1789762861.537767:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2396:x1876702067420544:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762861.539864:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.539903:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067420672 00000100:00100000:0.0:1789762861.539909:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067420672 00002000:00100000:0.0:1789762861.539942:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xb:0x0] (5242880->6291455) req@ffff9c37ee761c00 x1876702067420672/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762861.539951:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee761c00 with x1876702067420672 ext(5242880->6291455) 00002000:00010000:0.0:1789762861.539954:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ee761c00 00000100:00100000:0.0:1789762861.539963:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334779579150 time 334779610938 state 0x0 ref 1 00000100:00100000:0.0:1789762861.539986:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334779710938 time 334779610938 state 0x1 ref 1 00000100:00100000:0.0:1789762861.540008:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334779710938 time 334787086334 state 0x0 ref 1 00000100:00100000:0.0:1789762861.540031:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:2395:x1876702067420672:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762861.542065:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:2398:x1876702067417728:12345-192.168.203.37@tcp:4:dd.500 Request processed in 4804us (166910us total) trans 4294967799 rc 0/0 00000100:00100000:0.0:1789762861.542071:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 470 00002000:00100000:0.0:1789762861.542078:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xa:0x0] (4194304->5242879) req@ffff9c37d8cf6d80 x1876702067417728/t4294967799(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762861.542085:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf6d80 with x1876702067417728 ext(4194304->5242879) 00002000:00010000:0.0:1789762861.542088:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf6d80 00000100:00100000:0.0:1789762861.542942:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2396:x1876702067420544:12345-192.168.203.37@tcp:4:dd.0 Request processed in 5151us (5306us total) trans 4294967387 rc 0/0 00000100:00100000:0.0:1789762861.542949:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 5 00000100:00100000:0.0:1789762861.542955:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 334752788962 time 334784826762 state 0x2 ref 0 00002000:00100000:0.0:1789762861.542963:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0xb:0x0] (2097152->3145727) req@ffff9c37d8ccdc00 x1876702067420544/t4294967387(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762861.542970:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ccdc00 with x1876702067420544 ext(2097152->3145727) 00010000:00010000:0.0:1789762861.542976:0:8058:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 27 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.542988:0:8058:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 27 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762861.542994:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9c37d8ccdc00 00000100:00100000:3.0:1789762861.545656:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.545688:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067420800 00000100:00100000:0.0:1789762861.545693:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067420800 00010000:00010000:0.0:1789762861.545712:0:9878:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 27 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.545740:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 77 00000100:00100000:0.0:1789762861.545746:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c92850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2395:x1876702067420800:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762861.545757:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1789762861.545770:0:9878:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0xb:0x0] updating lvb size from disk: 0 -> 3145728 00002000:00010000:0.0:1789762861.545774:0:9878:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0xb:0x0] updating lvb mtime from disk: 0 -> 1789762861 00002000:00010000:0.0:1789762861.545778:0:9878:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0xb:0x0] updating lvb ctime from disk: 0 -> 1789762861 00010000:00010000:0.0:1789762861.545786:0:9878:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 27 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545801:0:9878:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 27 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545827:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000dd044021/0xdf5a7ac59f176080 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c84e expref: 27 pid: 8053 timeout: 434 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545841:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762861.545845:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762861.545852:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001c40f06e/0xdf5a7ac59f176087 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c855 expref: 26 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545861:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001c40f06e/0xdf5a7ac59f176087 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c855 expref: 26 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545879:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762861.545881:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762861.545886:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f176095 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c86a expref: 26 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545892:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f176095 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c86a expref: 26 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545896:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762861.545898:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762861.545901:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17609c lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c871 expref: 26 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545906:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17609c lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c871 expref: 26 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545911:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762861.545912:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762861.545944:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000020398f3e/0xdf5a7ac59f1760a3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c878 expref: 26 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545950:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000020398f3e/0xdf5a7ac59f1760a3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c878 expref: 26 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545956:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762861.545964:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762861.545969:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000081703a57/0xdf5a7ac59f1760aa lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c85c expref: 26 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545977:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000081703a57/0xdf5a7ac59f1760aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c85c expref: 26 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545983:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762861.545985:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762861.545989:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000071d3bc94/0xdf5a7ac59f1760b1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c87f expref: 26 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.545995:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000071d3bc94/0xdf5a7ac59f1760b1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c87f expref: 26 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.546025:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000071d3bc94/0xdf5a7ac59f1760b1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c87f expref: 26 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.546037:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074695296:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762861.546108:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000081703a57/0xdf5a7ac59f1760aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c85c expref: 26 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.546122:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90dc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074695424:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762861.546167:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000020398f3e/0xdf5a7ac59f1760a3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c878 expref: 26 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.546188:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074695552:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762861.546224:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17609c lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c871 expref: 26 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.546231:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074695680:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762861.546251:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f176095 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c86a expref: 26 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.546259:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2528700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074695808:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762861.546278:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001c40f06e/0xdf5a7ac59f176087 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c855 expref: 26 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762861.546284:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f58380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074695936:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762861.546299:0:9878:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f840 going to sleep for 11 seconds 00000100:00100000:1.0:1789762861.546487:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:2395:x1876702067420672:12345-192.168.203.37@tcp:4:dd.0 Request processed in 6456us (6621us total) trans 4294967800 rc 0/0 00000100:00100000:1.0:1789762861.546494:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 479 00000100:00100000:1.0:1789762861.546502:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334779710938 time 334787086334 state 0x2 ref 0 00002000:00100000:1.0:1789762861.546511:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xb:0x0] (5242880->6291455) req@ffff9c37ee761c00 x1876702067420672/t4294967800(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762861.546519:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ee761c00 with x1876702067420672 ext(5242880->6291455) 00002000:00010000:1.0:1789762861.546522:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ee761c00 00000100:00100000:0.0:1789762861.546882:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90d180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074695296:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762861.553770:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074695680:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762861.553789:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90dc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074695424:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762861.553808:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074695552:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762861.553824:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2528700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074695808:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762861.553830:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f58380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074695936:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762861.553864:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762861.553889:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c92850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2395:x1876702067420800:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 8143us (8236us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762861.553896:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 77 00000100:00100000:0.0:1789762861.568075:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6b9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074696064:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762861.568122:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762861.568166:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074696064 00000100:00100000:1.0:1789762861.568171:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074696064 00000100:00100000:1.0:1789762861.568207:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 319 00000100:00100000:1.0:1789762861.568215:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074696064:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762861.568242:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762861.568297:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074696064:12345-0@lo:41:osp-pre-1-0.0 Request processed in 83us (175us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.568303:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 319 00000100:00100000:0.0:1789762861.568338:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6b9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074696064:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762861.582376:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762861.582448:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067420928 00000100:00100000:1.0:1789762861.582455:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067420928 00002000:00100000:1.0:1789762861.582486:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0xb:0x0] (7340032->8388607) req@ffff9c37d1abb100 x1876702067420928/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762861.582496:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1abb100 with x1876702067420928 ext(7340032->8388607) 00002000:00010000:1.0:1789762861.582500:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d1abb100 00000100:00100000:1.0:1789762861.582514:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334779710938 time 334787086334 state 0x0 ref 1 00000100:00100000:1.0:1789762861.582545:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334787186334 time 334787086334 state 0x1 ref 1 00000100:00100000:1.0:1789762861.582566:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334787186334 time 334829643127 state 0x0 ref 1 00000100:00100000:1.0:1789762861.582576:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:2395:x1876702067420928:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762861.585964:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2395:x1876702067420928:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3383us (3586us total) trans 4294967801 rc 0/0 00000100:00100000:1.0:1789762861.585975:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 480 00000100:00100000:1.0:1789762861.585993:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334787186334 time 334829643127 state 0x2 ref 0 00002000:00100000:1.0:1789762861.586008:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0xb:0x0] (7340032->8388607) req@ffff9c37d1abb100 x1876702067420928/t4294967801(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762861.586017:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1abb100 with x1876702067420928 ext(7340032->8388607) 00002000:00010000:1.0:1789762861.586021:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d1abb100 00000100:00100000:1.0:1789762861.587322:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334884395573 time 334834395573 state 0x1 ref 7 00000100:00100000:1.0:1789762861.587340:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2398:x1876702067417856:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762861.590051:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2398:x1876702067417856:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2710us (208358us total) trans 4294967802 rc 0/0 00000100:00100000:1.0:1789762861.590063:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 471 00002000:00100000:1.0:1789762861.590074:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0xa:0x0] (5242880->6291455) req@ffff9c37d8cf7480 x1876702067417856/t4294967802(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762861.590085:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf7480 with x1876702067417856 ext(5242880->6291455) 00002000:00010000:1.0:1789762861.590091:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf7480 00000100:00100000:3.0:1789762861.590347:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.590404:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067421056 00000100:00100000:0.0:1789762861.590411:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067421056 00002000:00100000:0.0:1789762861.590441:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0xb:0x0] (7340032->8388607) req@ffff9c37d1ae2d80 x1876702067421056/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762861.590449:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1ae2d80 with x1876702067421056 ext(7340032->8388607) 00002000:00010000:0.0:1789762861.590453:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d1ae2d80 00000100:00100000:0.0:1789762861.590463:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334787186334 time 334829643127 state 0x0 ref 1 00000100:00100000:0.0:1789762861.590493:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334829743127 time 334829643127 state 0x1 ref 1 00000100:00100000:0.0:1789762861.590513:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334829743127 time 334837591777 state 0x0 ref 1 00000100:00100000:0.0:1789762861.590524:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2396:x1876702067421056:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762861.593429:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2396:x1876702067421056:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2905us (3081us total) trans 4294967388 rc 0/0 00000100:00100000:0.0:1789762861.593437:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 481 00000100:00100000:0.0:1789762861.593444:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334829743127 time 334837591777 state 0x2 ref 0 00002000:00100000:0.0:1789762861.593460:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0xb:0x0] (7340032->8388607) req@ffff9c37d1ae2d80 x1876702067421056/t4294967388(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762861.593467:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1ae2d80 with x1876702067421056 ext(7340032->8388607) 00002000:00010000:0.0:1789762861.593471:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d1ae2d80 00000100:00100000:3.0:1789762861.594083:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.594122:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067421184 00000100:00100000:2.0:1789762861.594128:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067421184 00002000:00100000:2.0:1789762861.594152:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0xb:0x0] (4194304->5242879) req@ffff9c37fd9f7b80 x1876702067421184/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762861.594160:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fd9f7b80 with x1876702067421184 ext(4194304->5242879) 00002000:00010000:2.0:1789762861.594163:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fd9f7b80 00000100:00100000:2.0:1789762861.594172:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334829743127 time 334837591777 state 0x0 ref 1 00000100:00100000:2.0:1789762861.594198:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334837691777 time 334837591777 state 0x1 ref 1 00000100:00100000:2.0:1789762861.594216:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334837691777 time 334841294342 state 0x0 ref 1 00000100:00100000:2.0:1789762861.594225:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2397:x1876702067421184:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762861.597036:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:2397:x1876702067421184:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2809us (2951us total) trans 4294967803 rc 0/0 00000100:00100000:2.0:1789762861.597045:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 482 00000100:00100000:2.0:1789762861.597055:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334837691777 time 334841294342 state 0x2 ref 0 00002000:00100000:2.0:1789762861.597067:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0xb:0x0] (4194304->5242879) req@ffff9c37fd9f7b80 x1876702067421184/t4294967803(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762861.597078:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fd9f7b80 with x1876702067421184 ext(4194304->5242879) 00002000:00010000:2.0:1789762861.597092:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fd9f7b80 00000100:00100000:3.0:1789762861.597328:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762861.597370:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067421312 00000100:00100000:1.0:1789762861.597377:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067421312 00002000:00100000:1.0:1789762861.597420:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0xb:0x0] (6291456->7340031) req@ffff9c37fd9f6680 x1876702067421312/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762861.597439:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fd9f6680 with x1876702067421312 ext(6291456->7340031) 00002000:00010000:1.0:1789762861.597445:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fd9f6680 00000100:00100000:1.0:1789762861.597457:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334837691777 time 334841294342 state 0x0 ref 1 00000100:00100000:1.0:1789762861.597483:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334841394342 time 334841294342 state 0x1 ref 1 00000100:00100000:1.0:1789762861.597506:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334841394342 time 334844584410 state 0x0 ref 1 00000100:00100000:1.0:1789762861.597515:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:2397:x1876702067421312:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762861.600909:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762861.600962:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067421440 00000100:00100000:1.0:1789762861.600968:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067421440 00002000:00100000:1.0:1789762861.600993:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xb:0x0] (3145728->4194303) req@ffff9c37d1ab8700 x1876702067421440/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762861.601000:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1ab8700 with x1876702067421440 ext(3145728->4194303) 00002000:00010000:1.0:1789762861.601004:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d1ab8700 00000100:00100000:0.0:1789762861.601022:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2397:x1876702067421312:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3497us (3684us total) trans 4294967804 rc 0/0 00000100:00100000:1.0:1789762861.601038:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334844684410 time 334844584410 state 0x1 ref 2 00000100:00100000:0.0:1789762861.601049:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 483 00002000:00100000:0.0:1789762861.601061:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0xb:0x0] (6291456->7340031) req@ffff9c37fd9f6680 x1876702067421312/t4294967804(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1789762861.601079:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334844684410 time 334848145142 state 0x0 ref 1 00002000:00100000:0.0:1789762861.601082:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fd9f6680 with x1876702067421312 ext(6291456->7340031) 00002000:00010000:0.0:1789762861.601091:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fd9f6680 00000100:00100000:3.0:1789762861.601097:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2397:x1876702067421440:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762861.604969:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.605005:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067421568 00000100:00100000:0.0:1789762861.605011:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067421568 00002000:00100000:0.0:1789762861.605033:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0xb:0x0] (1048576->2097151) req@ffff9c37ff6b9c00 x1876702067421568/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762861.605042:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff6b9c00 with x1876702067421568 ext(1048576->2097151) 00002000:00010000:0.0:1789762861.605046:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff6b9c00 00000100:00100000:1.0:1789762861.605059:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762861.605086:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067421696 00000100:00100000:0.0:1789762861.605092:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334848245142 time 334848145142 state 0x1 ref 2 00000100:00100000:3.0:1789762861.605093:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067421696 00000100:00100000:1.0:1789762861.605111:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334848245142 time 334852185317 state 0x0 ref 2 00002000:00100000:3.0:1789762861.605120:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0xb:0x0] (5242880->6291455) req@ffff9c37ff6bb800 x1876702067421696/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1789762861.605128:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:2398:x1876702067421568:12345-192.168.203.37@tcp:4:dd.0 00002000:00100000:3.0:1789762861.605140:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff6bb800 with x1876702067421696 ext(5242880->6291455) 00002000:00010000:3.0:1789762861.605146:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ff6bb800 00000100:00100000:3.0:1789762861.605172:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334852285317 time 334852185317 state 0x1 ref 3 00000100:00100000:0.0:1789762861.605227:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334852285317 time 334852303073 state 0x0 ref 3 00000100:00100000:0.0:1789762861.605240:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2395:x1876702067421696:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762861.605299:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2397:x1876702067421440:12345-192.168.203.37@tcp:4:dd.0 Request processed in 4205us (4391us total) trans 4294967805 rc 0/0 00000100:00100000:2.0:1789762861.605307:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 484 00002000:00100000:2.0:1789762861.605315:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0xb:0x0] (3145728->4194303) req@ffff9c37d1ab8700 x1876702067421440/t4294967805(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762861.605322:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1ab8700 with x1876702067421440 ext(3145728->4194303) 00002000:00010000:2.0:1789762861.605326:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d1ab8700 00000100:00100000:0.0:1789762861.608999:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2398:x1876702067421568:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3874us (4030us total) trans 4294967806 rc 0/0 00000100:00100000:0.0:1789762861.609009:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 485 00002000:00100000:0.0:1789762861.609019:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xb:0x0] (1048576->2097151) req@ffff9c37ff6b9c00 x1876702067421568/t4294967806(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762861.609030:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff6b9c00 with x1876702067421568 ext(1048576->2097151) 00002000:00010000:0.0:1789762861.609036:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff6b9c00 00000100:00100000:2.0:1789762861.612288:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.612340:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067421824 00000100:00100000:0.0:1789762861.612349:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067421824 00000100:00100000:2.0:1789762861.612356:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762861.612399:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067421952 00002000:00100000:0.0:1789762861.612403:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0xb:0x0] (2097152->3145727) req@ffff9c37d1abb480 x1876702067421824/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1789762861.612405:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067421952 00002000:00100000:0.0:1789762861.612418:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1abb480 with x1876702067421824 ext(2097152->3145727) 00002000:00010000:0.0:1789762861.612424:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d1abb480 00002000:00100000:3.0:1789762861.612429:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0xb:0x0] (1048576->2097151) req@ffff9c37fdfe0e00 x1876702067421952/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1789762861.612475:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334852403073 time 334852303073 state 0x1 ref 2 00000100:00100000:2.0:1789762861.612493:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334852403073 time 334859566058 state 0x0 ref 2 00000100:00100000:2.0:1789762861.612507:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2396:x1876702067421824:12345-192.168.203.37@tcp:4:dd.0 00002000:00100000:3.0:1789762861.612777:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfe0e00 with x1876702067421952 ext(1048576->2097151) 00002000:00010000:3.0:1789762861.612781:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fdfe0e00 00000100:00100000:3.0:1789762861.612817:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334859666058 time 334859566058 state 0x1 ref 3 00000100:00100000:2.0:1789762861.612842:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334859666058 time 334859913990 state 0x0 ref 3 00000100:00100000:2.0:1789762861.612857:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:2396:x1876702067421952:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762861.616559:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2395:x1876702067421696:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11320us (11498us total) trans 4294967389 rc 0/0 00000100:00100000:1.0:1789762861.616568:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 486 00002000:00100000:1.0:1789762861.616575:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0xb:0x0] (5242880->6291455) req@ffff9c37ff6bb800 x1876702067421696/t4294967389(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762861.616582:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff6bb800 with x1876702067421696 ext(5242880->6291455) 00002000:00010000:1.0:1789762861.616586:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ff6bb800 00000100:00100000:2.0:1789762861.617564:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762861.617656:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067422080 00000100:00100000:3.0:1789762861.617662:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067422080 00002000:00100000:3.0:1789762861.618898:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0xb:0x0] (4194304->5242879) req@ffff9c37fdfe1f80 x1876702067422080/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762861.618907:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfe1f80 with x1876702067422080 ext(4194304->5242879) 00002000:00010000:3.0:1789762861.618911:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fdfe1f80 00000100:00100000:3.0:1789762861.619034:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334860013990 time 334859913990 state 0x1 ref 3 00000100:00100000:0.0:1789762861.619100:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334860013990 time 334866171127 state 0x0 ref 3 00000100:00100000:0.0:1789762861.619123:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2398:x1876702067422080:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762861.620080:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.620136:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067422336 00000100:00100000:0.0:1789762861.620149:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067422336 00000100:00100000:0.0:1789762861.620186:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 71 00000100:00100000:0.0:1789762861.620194:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:14536:x1876702067422336:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:0.0:1789762861.620495:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:14536:x1876702067422336:12345-192.168.203.37@tcp:35:dd.0 Request processed in 301us (417us total) trans 4294967409 rc 0/0 00000100:00100000:0.0:1789762861.620509:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 71 00000100:00100000:2.0:1789762861.620804:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2396:x1876702067421952:12345-192.168.203.37@tcp:4:dd.0 Request processed in 7949us (8448us total) trans 4294967390 rc 0/0 00000100:00100000:0.0:1789762861.620812:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2396:x1876702067421824:12345-192.168.203.37@tcp:4:dd.0 Request processed in 8307us (8525us total) trans 4294967807 rc 0/0 00000100:00100000:2.0:1789762861.620816:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 488 00000100:00100000:0.0:1789762861.620826:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 487 00002000:00100000:2.0:1789762861.620828:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0xb:0x0] (1048576->2097151) req@ffff9c37fdfe0e00 x1876702067421952/t4294967390(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762861.620836:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0xb:0x0] (2097152->3145727) req@ffff9c37d1abb480 x1876702067421824/t4294967807(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762861.620854:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1abb480 with x1876702067421824 ext(2097152->3145727) 00002000:00100000:2.0:1789762861.620861:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfe0e00 with x1876702067421952 ext(1048576->2097151) 00002000:00010000:2.0:1789762861.620867:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fdfe0e00 00002000:00010000:0.0:1789762861.620870:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d1abb480 00000100:00100000:1.0:1789762861.621794:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762861.621828:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067422464 00000100:00100000:3.0:1789762861.621833:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067422464 00002000:00100000:3.0:1789762861.621858:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0xb:0x0] (3145728->4194303) req@ffff9c37d8ca5c00 x1876702067422464/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762861.622009:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ca5c00 with x1876702067422464 ext(3145728->4194303) 00002000:00010000:3.0:1789762861.622015:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ca5c00 00000100:00100000:1.0:1789762861.622045:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762861.622065:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067422592 00000100:00100000:1.0:1789762861.622069:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067422592 00002000:00100000:1.0:1789762861.622089:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0xb:0x0] (6291456->7340031) req@ffff9c37f60a1180 x1876702067422592/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762861.622098:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f60a1180 with x1876702067422592 ext(6291456->7340031) 00002000:00010000:1.0:1789762861.622102:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f60a1180 00000100:00100000:1.0:1789762861.622133:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334866271127 time 334866171127 state 0x1 ref 3 00000100:00100000:3.0:1789762861.622151:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334866271127 time 334866171127 state 0x1 ref 3 00000100:00100000:2.0:1789762861.622169:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 334869342170 time 334869242170 state 0x1 ref 3 00000100:00100000:2.0:1789762861.622188:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2395:x1876702067422592:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762861.622189:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 1 deadline 334869342170 time 334869259177 state 0x0 ref 3 00000100:00100000:1.0:1789762861.622210:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ca5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2396:x1876702067422464:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762861.622505:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2398:x1876702067422080:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3390us (4942us total) trans 4294967391 rc 0/0 00000100:00100000:0.0:1789762861.622512:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 489 00002000:00100000:0.0:1789762861.622519:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0xb:0x0] (4194304->5242879) req@ffff9c37fdfe1f80 x1876702067422080/t4294967391(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762861.622525:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfe1f80 with x1876702067422080 ext(4194304->5242879) 00002000:00010000:0.0:1789762861.622529:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fdfe1f80 00000100:00100000:1.0:1789762861.625449:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2395:x1876702067422592:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3264us (3402us total) trans 4294967392 rc 0/0 00000100:00100000:1.0:1789762861.625459:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 490 00002000:00100000:1.0:1789762861.625467:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0xb:0x0] (6291456->7340031) req@ffff9c37f60a1180 x1876702067422592/t4294967392(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762861.625478:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f60a1180 with x1876702067422592 ext(6291456->7340031) 00002000:00010000:1.0:1789762861.625483:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f60a1180 00000100:00100000:1.0:1789762861.629004:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ca5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2396:x1876702067422464:12345-192.168.203.37@tcp:4:dd.0 Request processed in 6784us (7198us total) trans 4294967393 rc 0/0 00000100:00100000:1.0:1789762861.629018:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 491 00000100:00100000:1.0:1789762861.629038:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 1 deadline 334869342170 time 334869259177 state 0x2 ref 0 00002000:00100000:1.0:1789762861.629054:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0xb:0x0] (3145728->4194303) req@ffff9c37d8ca5c00 x1876702067422464/t4294967393(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762861.629062:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8ca5c00 with x1876702067422464 ext(3145728->4194303) 00002000:00010000:1.0:1789762861.629069:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8ca5c00 00000100:00100000:0.0:1789762861.637423:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334934488240 time 334884488240 state 0x1 ref 6 00000100:00100000:0.0:1789762861.637448:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2398:x1876702067417984:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762861.640548:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2398:x1876702067417984:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3109us (251336us total) trans 4294967808 rc 0/0 00000100:00100000:1.0:1789762861.640557:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 472 00002000:00100000:1.0:1789762861.640567:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0xa:0x0] (6291456->7340031) req@ffff9c37d8cf5f80 x1876702067417984/t4294967808(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762861.640575:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf5f80 with x1876702067417984 ext(6291456->7340031) 00002000:00010000:1.0:1789762861.640581:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf5f80 00000100:00100000:3.0:1789762861.687516:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 334984575949 time 334934575949 state 0x1 ref 5 00000100:00100000:3.0:1789762861.687548:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2397:x1876702067418112:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762861.690798:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2397:x1876702067418112:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3258us (294956us total) trans 4294967809 rc 0/0 00000100:00100000:3.0:1789762861.690808:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 473 00002000:00100000:3.0:1789762861.690821:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0xa:0x0] (7340032->8388607) req@ffff9c37d8cf7b80 x1876702067418112/t4294967809(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762861.690838:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf7b80 with x1876702067418112 ext(7340032->8388607) 00002000:00010000:3.0:1789762861.690845:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf7b80 00010000:00010000:0.0:1789762861.696031:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762861.737602:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335034661652 time 334984661652 state 0x1 ref 4 00000100:00100000:3.0:1789762861.737630:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2397:x1876702067418240:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762861.740874:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2397:x1876702067418240:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3253us (337489us total) trans 4294967394 rc 0/0 00000100:00100000:3.0:1789762861.740881:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 474 00002000:00100000:3.0:1789762861.740892:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0xa:0x0] (4194304->5242879) req@ffff9c37d8cf4000 x1876702067418240/t4294967394(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762861.740900:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf4000 with x1876702067418240 ext(4194304->5242879) 00002000:00010000:3.0:1789762861.740907:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf4000 00000100:00100000:3.0:1789762861.787684:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335084744864 time 335034744864 state 0x1 ref 3 00000100:00100000:3.0:1789762861.787722:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2397:x1876702067418368:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762861.790886:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2397:x1876702067418368:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3183us (380949us total) trans 4294967395 rc 0/0 00000100:00100000:3.0:1789762861.790894:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 475 00002000:00100000:3.0:1789762861.790905:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0xa:0x0] (5242880->6291455) req@ffff9c37f608b100 x1876702067418368/t4294967395(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762861.790914:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f608b100 with x1876702067418368 ext(5242880->6291455) 00002000:00010000:3.0:1789762861.790953:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f608b100 00000100:00100000:0.0:1789762861.837770:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335134834387 time 335084834387 state 0x1 ref 2 00000100:00100000:0.0:1789762861.837795:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2397:x1876702067418496:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762861.840859:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2397:x1876702067418496:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3072us (399111us total) trans 4294967396 rc 0/0 00000100:00100000:3.0:1789762861.840868:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 476 00002000:00100000:3.0:1789762861.840878:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0xa:0x0] (6291456->7340031) req@ffff9c37f6088a80 x1876702067418496/t4294967396(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762861.840886:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f6088a80 with x1876702067418496 ext(6291456->7340031) 00002000:00010000:3.0:1789762861.840892:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f6088a80 00000100:00100000:1.0:1789762861.887996:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335134834387 time 335135046218 state 0x0 ref 1 00000100:00100000:1.0:1789762861.888040:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2397:x1876702067418624:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762861.895302:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2397:x1876702067418624:12345-192.168.203.37@tcp:4:dd.500 Request processed in 7278us (446850us total) trans 4294967397 rc 0/0 00000100:00100000:1.0:1789762861.895314:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 477 00000100:00100000:1.0:1789762861.895324:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335134834387 time 335135046218 state 0x2 ref 0 00002000:00100000:1.0:1789762861.895339:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0xa:0x0] (7340032->8388607) req@ffff9c37d8cf4700 x1876702067418624/t4294967397(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/448 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762861.895358:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf4700 with x1876702067418624 ext(7340032->8388607) 00002000:00010000:1.0:1789762861.895366:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf4700 00000100:00100000:2.0:1789762861.936055:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.936105:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762861.936121:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067422720 00000100:00100000:1.0:1789762861.936133:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067422720 00000100:00100000:3.0:1789762861.936143:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067422848 00000100:00100000:3.0:1789762861.936151:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067422848 00000100:00100000:1.0:1789762861.936202:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 47 00000100:00100000:1.0:1789762861.936219:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:6823:x1876702067422720:12345-192.168.203.37@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1789762861.936219:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 48 00000100:00100000:3.0:1789762861.936236:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:6358:x1876702067422848:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762861.936239:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 7 locks, starting at 0 00010000:00010000:3.0:1789762861.936251:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00002000:00010000:1.0:1789762861.936263:0:6404:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0xa:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:1.0:1789762861.936267:0:6404:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0xa:0x0] updating lvb mtime from disk: 0 -> 1789762861 00002000:00010000:1.0:1789762861.936269:0:6404:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0xa:0x0] updating lvb ctime from disk: 0 -> 1789762861 00002000:00010000:3.0:1789762861.936269:0:6406:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xa:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:3.0:1789762861.936276:0:6406:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xa:0x0] updating lvb mtime from disk: 0 -> 1789762861 00002000:00010000:1.0:1789762861.936277:0:6404:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000400:0xa:0x0] updating lvb blocks from disk: 1 -> 2051 00002000:00010000:3.0:1789762861.936283:0:6406:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xa:0x0] updating lvb ctime from disk: 0 -> 1789762861 00010000:00010000:1.0:1789762861.936310:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000012026acc/0xdf5a7ac59f176056 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c840 expref: 42 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.936314:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f176048 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c839 expref: 39 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762861.936354:0:6404:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0xb:0x0] updating lvb size from disk: 3145728 -> 8388608 00002000:00010000:3.0:1789762861.936361:0:6406:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xb:0x0] updating lvb size from disk: 0 -> 8388608 00010000:00010000:1.0:1789762861.936366:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000081703a57/0xdf5a7ac59f1760aa lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c85c expref: 41 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762861.936367:0:6406:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xb:0x0] updating lvb mtime from disk: 0 -> 1789762861 00002000:00010000:3.0:1789762861.936374:0:6406:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xb:0x0] updating lvb ctime from disk: 0 -> 1789762861 00010000:00010000:1.0:1789762861.936407:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f176095 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c86a expref: 40 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.936418:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000020398f3e/0xdf5a7ac59f1760a3 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 6 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c878 expref: 39 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.936431:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001c40f06e/0xdf5a7ac59f176087 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 5 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c855 expref: 38 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.936431:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f17608e lrc: 0/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c863 expref: 38 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.936440:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17609c lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 4 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c871 expref: 37 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.936449:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000071d3bc94/0xdf5a7ac59f1760b1 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 3 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c87f expref: 36 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762861.936463:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762861.936502:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762861.936516:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:6823:x1876702067422720:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 305us (475us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762861.936524:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 47 00000100:00100000:3.0:1789762861.936546:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:6358:x1876702067422848:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 316us (441us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762861.936553:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 48 00000100:00100000:0.0:1789762861.973132:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762861.973182:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067423104 00000100:00100000:3.0:1789762861.973190:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067423104 00000100:00100000:3.0:1789762861.973233:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 264 00000100:00100000:3.0:1789762861.973240:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:14562:x1876702067423104:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762861.973263:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762861.973281:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d400/0xdf5a7ac59f1760b8 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c886 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.973436:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d600/0xdf5a7ac59f1760bf lrc: 3/1,0 mode: --/PR res: [0x240000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.973451:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d600/0xdf5a7ac59f1760bf lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.973462:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d600/0xdf5a7ac59f1760bf lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762861.973472:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x12:0x0]: returning size 16777216 00010000:00010000:3.0:1789762861.973603:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d400/0xdf5a7ac59f1760b8 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c886 expref: 25 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.973618:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d600/0xdf5a7ac59f1760bf lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c886 expref: 24 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.973625:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x12:0x0].0x0 (ffff9c37e125ad00) 00010000:00010000:3.0:1789762861.973628:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762861.973641:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d273d600, rc 0) 00000100:00100000:3.0:1789762861.973683:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:14562:x1876702067423104:12345-192.168.203.37@tcp:101:dd.0 Request processed in 443us (561us total) trans 4294967410 rc 0/0 00000100:00100000:3.0:1789762861.973689:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 264 00000100:00100000:0.0:1789762861.974732:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762861.974771:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067423360 00000100:00100000:3.0:1789762861.974776:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067423360 00000100:00100000:3.0:1789762861.974803:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 265 00000100:00100000:3.0:1789762861.974809:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:14563:x1876702067423360:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762861.974817:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762861.974826:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d800/0xdf5a7ac59f1760cd lrc: 2/0,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c88d expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.974866:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273de00/0xdf5a7ac59f1760d4 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.974878:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273de00/0xdf5a7ac59f1760d4 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.974888:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273de00/0xdf5a7ac59f1760d4 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762861.974898:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x12:0x0]: returning size 16777216 00010000:00010000:3.0:1789762861.974976:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d800/0xdf5a7ac59f1760cd lrc: 0/0,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c88d expref: 27 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.974988:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273de00/0xdf5a7ac59f1760d4 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c88d expref: 26 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762861.974994:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x12:0x0].0x0 (ffff9c37e125ad00) 00010000:00010000:3.0:1789762861.975002:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762861.975004:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d273de00, rc 0) 00000100:00100000:3.0:1789762861.975030:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:14563:x1876702067423360:12345-192.168.203.37@tcp:101:dd.0 Request processed in 222us (300us total) trans 4294967411 rc 0/0 00000100:00100000:3.0:1789762861.975036:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 265 00000100:00100000:1.0:1789762861.975267:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762861.975316:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.975319:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067423616 00000100:00100000:0.0:1789762861.975327:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067423616 00000100:00100000:2.0:1789762861.975365:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067423744 00000100:00100000:2.0:1789762861.975372:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067423744 00000100:00100000:0.0:1789762861.975375:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 266 00000100:00100000:0.0:1789762861.975405:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:14564:x1876702067423616:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762861.975419:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762861.975432:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f1760e2 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c894 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762861.975440:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 49 00000100:00100000:2.0:1789762861.975458:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2396:x1876702067423744:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762861.975471:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762861.975488:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d600/0xdf5a7ac59f1760bf lrc: 0/0,0 mode: --/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c886 expref: 28 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.975492:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769800/0xdf5a7ac59f1760e9 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.975509:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769800/0xdf5a7ac59f1760e9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.975521:0:6405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x12:0x0].0x0 (ffff9c37e125ad00) 00010000:00010000:0.0:1789762861.975522:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769800/0xdf5a7ac59f1760e9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.975530:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00200000:0.0:1789762861.975537:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x12:0x0]: returning size 16777216 00000100:00100000:2.0:1789762861.975568:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2396:x1876702067423744:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 115us (250us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762861.975578:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 49 00010000:00010000:0.0:1789762861.975621:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f1760e2 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c894 expref: 28 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.975639:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769800/0xdf5a7ac59f1760e9 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c894 expref: 27 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.975648:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x12:0x0].0x0 (ffff9c37e125ad00) 00010000:00010000:0.0:1789762861.975651:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762861.975661:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769800, rc 0) 00000100:00100000:0.0:1789762861.975699:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:14564:x1876702067423616:12345-192.168.203.37@tcp:101:dd.0 Request processed in 298us (432us total) trans 4294967412 rc 0/0 00000100:00100000:0.0:1789762861.975707:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 266 00000100:00100000:1.0:1789762861.976002:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.976030:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067423872 00000100:00100000:0.0:1789762861.976035:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067423872 00000100:00100000:0.0:1789762861.976065:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 168 00000100:00100000:0.0:1789762861.976072:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:14562:x1876702067423872:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762861.976082:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762861.976093:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f1760f7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c89b expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762861.976114:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xa:0x0] initial LVB size: 8388608, mtime: 0x6aad9d2d, atime: 0x0, ctime: 0x6aad9d2d, blocks: 0x1 00010000:00010000:0.0:1789762861.976133:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xa:0x0].0x0 since=335s v=(0 0) 00010000:00010000:0.0:1789762861.976140:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xa:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762861.976153:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f1760f7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c89b expref: 38 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762861.976170:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f1760f7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c89b expref: 38 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762861.976179:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:0.0:1789762861.976181:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768800, rc 0) 00000100:00100000:0.0:1789762861.976216:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:14562:x1876702067423872:12345-192.168.203.37@tcp:101:dd.0 Request processed in 145us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762861.976217:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067424128 00000100:00100000:0.0:1789762861.976226:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 168 00000100:00100000:2.0:1789762861.976226:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067424128 00000100:00100000:2.0:1789762861.976272:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 72 00000100:00100000:2.0:1789762861.976283:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfeed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:14563:x1876702067424128:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:2.0:1789762861.976430:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfeed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:14563:x1876702067424128:12345-192.168.203.37@tcp:35:dd.0 Request processed in 148us (251us total) trans 4294967413 rc 0/0 00000100:00100000:2.0:1789762861.976439:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 72 00000100:00100000:1.0:1789762861.976619:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.976661:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067424256 00000100:00100000:2.0:1789762861.976667:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067424256 00000100:00100000:2.0:1789762861.976701:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 50 00000100:00100000:2.0:1789762861.976711:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2397:x1876702067424256:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762861.976720:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762861.976735:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273de00/0xdf5a7ac59f1760d4 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c88d expref: 28 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.976750:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x12:0x0].0x0 (ffff9c37e125ad00) 00010000:00010000:2.0:1789762861.976755:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762861.976788:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2397:x1876702067424256:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 77us (166us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762861.976789:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.976801:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 50 00000100:00100000:2.0:1789762861.976820:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067424384 00000100:00100000:2.0:1789762861.976825:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067424384 00000100:00100000:2.0:1789762861.976864:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 51 00000100:00100000:2.0:1789762861.976873:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2397:x1876702067424384:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762861.976881:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762861.976893:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769800/0xdf5a7ac59f1760e9 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c894 expref: 27 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762861.976906:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x12:0x0].0x0 (ffff9c37e125ad00) 00010000:00010000:2.0:1789762861.976910:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762861.976970:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2397:x1876702067424384:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 99us (213us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762861.976980:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 51 00000100:00100000:1.0:1789762861.977169:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762861.977199:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067424640 00000100:00100000:0.0:1789762861.977204:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067424640 00000100:00100000:0.0:1789762861.977237:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 73 00000100:00100000:0.0:1789762861.977244:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfefb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:14564:x1876702067424640:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:0.0:1789762861.977329:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfefb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:14564:x1876702067424640:12345-192.168.203.37@tcp:35:dd.0 Request processed in 85us (160us total) trans 4294967414 rc 0/0 00000100:00100000:0.0:1789762861.977334:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 73 00000100:00100000:1.0:1789762861.980817:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762861.980866:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067424768 00000100:00100000:2.0:1789762861.980872:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067424768 00002000:00100000:2.0:1789762861.980906:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xa:0x0] (0->1048575) req@ffff9c37fcfed880 x1876702067424768/t0(0) o3->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/0 e 0 to 0 dl 1789762872 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762861.980941:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fcfed880 with x1876702067424768 ext(0->1048575) 00002000:00010000:2.0:1789762861.980946:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fcfed880 00000100:00100000:2.0:1789762861.980967:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314925816443 time 314925912621 state 0x0 ref 1 00000100:00100000:2.0:1789762861.980978:0:8059:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 6: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c36c2126400 gen 0 rate 10 depth 3 token 0 deadline 314925816443 time 314925912621 state 0x0 ref 1 00000100:00100000:2.0:1789762861.981017:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 3 deadline 335228189611 time 335228089611 state 0x1 ref 1 00000100:00100000:2.0:1789762861.981045:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 335228189611 time 335228120661 state 0x0 ref 1 00000100:00100000:2.0:1789762861.981059:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2397:x1876702067424768:12345-192.168.203.37@tcp:3:dd.0 00000100:00100000:2.0:1789762861.985510:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2397:x1876702067424768:12345-192.168.203.37@tcp:3:dd.0 Request processed in 4451us (4693us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762861.985522:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 492 00000100:00100000:2.0:1789762861.985533:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 335228189611 time 335228120661 state 0x2 ref 0 00002000:00100000:2.0:1789762861.985545:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xa:0x0] (0->1048575) req@ffff9c37fcfed880 x1876702067424768/t0(0) o3->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:642/0 lens 488/440 e 0 to 0 dl 1789762872 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762861.985565:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fcfed880 with x1876702067424768 ext(0->1048575) 00002000:00010000:2.0:1789762861.985570:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fcfed880 00000100:00100000:1.0:1789762862.012643:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.012702:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067425024 00000100:00100000:2.0:1789762862.012720:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067425024 00000100:00100000:2.0:1789762862.012773:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 74 00000100:00100000:2.0:1789762862.012785:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:14577:x1876702067425024:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:2.0:1789762862.013046:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:14577:x1876702067425024:12345-192.168.203.37@tcp:35:dd.0 Request processed in 264us (408us total) trans 4294967415 rc 0/0 00000100:00100000:2.0:1789762862.013056:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 74 00000100:00100000:1.0:1789762862.022581:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.022639:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067425152 00000100:00100000:2.0:1789762862.022645:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067425152 00000100:00100000:2.0:1789762862.022682:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 267 00000100:00100000:2.0:1789762862.022690:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:14578:x1876702067425152:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:2.0:1789762862.022705:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762862.022732:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f1760fe lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8a2 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.022795:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6200/0xdf5a7ac59f176105 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.022814:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6200/0xdf5a7ac59f176105 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.022828:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6200/0xdf5a7ac59f176105 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762862.022857:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x11:0x0]: returning size 16384 00000004:00200000:2.0:1789762862.022871:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762862.022891:0:6419:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6200/0xdf5a7ac59f176105 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.022938:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6600/0xdf5a7ac59f1760fe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8a2 expref: 30 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.022963:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6200/0xdf5a7ac59f176105 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8a2 expref: 29 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.022974:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:2.0:1789762862.022977:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762862.022981:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc6200, rc 0) 00000100:00100000:2.0:1789762862.023015:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:14578:x1876702067425152:12345-192.168.203.37@tcp:101:rm.0 Request processed in 325us (434us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762862.023023:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 267 00000100:00100000:3.0:1789762862.024190:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762862.024255:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067425408 00000100:00100000:0.0:1789762862.024262:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067425408 00000100:00100000:0.0:1789762862.024301:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 268 00000100:00100000:0.0:1789762862.024312:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:14578:x1876702067425408:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:0.0:1789762862.024322:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762862.024336:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17610c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8a9 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762862.024404:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x11:0x0]: returning size 16384 00010000:00010000:0.0:1789762862.024492:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17610c lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8a9 expref: 31 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.024505:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37e125af00) 00010000:00010000:0.0:1789762862.024508:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762862.024513:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17610c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8a9 expref: 31 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.024527:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769200, rc 0) 00000100:00100000:0.0:1789762862.024563:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:14578:x1876702067425408:12345-192.168.203.37@tcp:101:rm.0 Request processed in 254us (374us total) trans 4294967416 rc 301/301 00000100:00100000:0.0:1789762862.024572:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 268 00000100:00100000:1.0:1789762862.025957:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762862.025993:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067425536 00000100:00100000:0.0:1789762862.026008:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067425536 00000100:00100000:0.0:1789762862.026052:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 75 00000100:00100000:0.0:1789762862.026064:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:14578:x1876702067425536:12345-192.168.203.37@tcp:37:rm.0 00000100:00100000:0.0:1789762862.027757:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:14578:x1876702067425536:12345-192.168.203.37@tcp:37:rm.0 Request processed in 1694us (1798us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762862.027766:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 75 00000100:00100000:3.0:1789762862.028979:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.029021:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067425792 00000100:00100000:2.0:1789762862.029026:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067425792 00000100:00100000:2.0:1789762862.029054:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 76 00000100:00100000:2.0:1789762862.029063:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbec50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:14578:x1876702067425792:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:2.0:1789762862.029166:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbec50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:14578:x1876702067425792:12345-192.168.203.37@tcp:35:rm.0 Request processed in 103us (187us total) trans 4294967417 rc 0/0 00000100:00100000:2.0:1789762862.029176:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 76 00000100:00100000:1.0:1789762862.029975:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762862.030019:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067426048 00000100:00100000:3.0:1789762862.030026:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067426048 00000100:00100000:3.0:1789762862.030058:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 269 00000100:00100000:3.0:1789762862.030066:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:14578:x1876702067426048:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:3.0:1789762862.030076:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762862.030086:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273da00/0xdf5a7ac59f17611a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8b0 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.030126:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273ca00/0xdf5a7ac59f176121 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.030134:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273ca00/0xdf5a7ac59f176121 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x11:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.030141:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273ca00/0xdf5a7ac59f176121 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x11:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762862.030147:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x11:0x0]: returning size 16384 00010000:00010000:3.0:1789762862.030204:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273da00/0xdf5a7ac59f17611a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8b0 expref: 34 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.030230:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273ca00/0xdf5a7ac59f176121 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x11:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8b0 expref: 33 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.030236:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:3.0:1789762862.030238:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1789762862.030240:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d273ca00, rc 0) 00000100:00100000:3.0:1789762862.030268:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:14578:x1876702067426048:12345-192.168.203.37@tcp:101:rm.0 Request processed in 203us (293us total) trans 4294967418 rc 0/0 00000100:00100000:3.0:1789762862.030272:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 269 00000100:00100000:1.0:1789762862.031028:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762862.031058:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067426176 00000100:00100000:3.0:1789762862.031062:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067426176 00000100:00100000:3.0:1789762862.031087:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 270 00000100:00100000:3.0:1789762862.031092:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a11f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14578:x1876702067426176:12345-192.168.203.37@tcp:41:rm.0 00000004:00000024:3.0:1789762862.031133:0:13389:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7719616512 avail 7715422208 00000100:00100000:3.0:1789762862.031167:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a11f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14578:x1876702067426176:12345-192.168.203.37@tcp:41:rm.0 Request processed in 75us (140us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762862.031172:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 270 00000100:00100000:2.0:1789762862.031936:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762862.031962:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067426432 00000100:00100000:3.0:1789762862.031967:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067426432 00000100:00100000:3.0:1789762862.031994:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 271 00000100:00100000:3.0:1789762862.032003:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:14578:x1876702067426432:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:3.0:1789762862.032032:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1789762862.032045:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f17603a lrc: 0/0,0 mode: --/PR res: [0x240000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c832 expref: 33 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.032064:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x12:0x0].0x0 (ffff9c37e125ad00) 00010000:00010000:3.0:1789762862.032073:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6200/0xdf5a7ac59f176105 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8a2 expref: 32 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.032080:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:3.0:1789762862.032082:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762862.032092:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c400/0xdf5a7ac59f17612f lrc: 3/0,1 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.032099:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c400/0xdf5a7ac59f17612f lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.032104:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c400/0xdf5a7ac59f17612f lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.032111:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d000/0xdf5a7ac59f176136 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x11:0x0].0x73550ecb bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.032116:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d000/0xdf5a7ac59f176136 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.032121:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d000/0xdf5a7ac59f176136 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.032179:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17613d lrc: 3/0,1 mode: --/EX res: [0x240000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.032184:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17613d lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.032197:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17613d lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789762862.033004:0:7744:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].10, 64 off8832 00000004:00080000:3.0:1789762862.033017:0:7744:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 335280098590ns, next at 336280095438ns, rc = 0 00000040:00080000:3.0:1789762862.033087:0:7744:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].10, 64 off8832 00000004:00080000:3.0:1789762862.033093:0:7744:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 335280174732ns, next at 336280173570ns, rc = 0 00010000:00010000:3.0:1789762862.033172:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c000/0xdf5a7ac59f176144 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033180:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c000/0xdf5a7ac59f176144 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033186:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c000/0xdf5a7ac59f176144 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033192:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c000/0xdf5a7ac59f176144 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033198:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c000/0xdf5a7ac59f176144 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033202:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c000/0xdf5a7ac59f176144 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033218:0:7744:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762862.033228:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17613d lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033232:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17613d lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.033244:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762862.033246:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17613d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033255:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762862.033257:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c000/0xdf5a7ac59f176144 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033271:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17613d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033287:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17613d lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033294:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x12:0x0].0x0 (ffff9c37c72ba800) 00010000:00010000:3.0:1789762862.033299:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789762862.033304:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17613d lrc: 2/0,0 mode: --/EX res: [0x240000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.033308:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762862.033312:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17613d lrc: 0/0,0 mode: --/EX res: [0x240000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.033316:0:7217:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c000/0xdf5a7ac59f176144 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.033327:0:7217:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c000/0xdf5a7ac59f176144 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033330:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c400/0xdf5a7ac59f17612f lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.033343:0:7217:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x12:0x0].0x0 (ffff9c37c72ba800) 00010000:00010000:3.0:1789762862.033343:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c400/0xdf5a7ac59f17612f lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.033358:0:7217:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789762862.033366:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c400/0xdf5a7ac59f17612f lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.033367:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c000/0xdf5a7ac59f176144 lrc: 1/0,0 mode: --/PW res: [0x240000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033394:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c400/0xdf5a7ac59f17612f lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.033398:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c000/0xdf5a7ac59f176144 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033402:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762862.033425:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c400/0xdf5a7ac59f17612f lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.033427:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762862.033431:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762862.033435:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762862.033438:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c400/0xdf5a7ac59f17612f lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033450:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:3.0:1789762862.033454:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762862.033457:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c400/0xdf5a7ac59f17612f lrc: 3/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033465:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273c400/0xdf5a7ac59f17612f lrc: 0/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762862.033476:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000051653996 reply state 00000000bbe7ec15 transno 4294967419 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d000/0xdf5a7ac59f176136 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033486:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d000/0xdf5a7ac59f176136 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.033491:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x73550ecb (ffff9c37e125ae00) 00010000:00010000:3.0:1789762862.033493:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1789762862.033495:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762862.033528:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:14578:x1876702067426432:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1528us (1612us total) trans 4294967419 rc 0/0 00000100:00100000:3.0:1789762862.033534:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 271 00000100:00100000:1.0:1789762862.035255:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762862.035296:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067426688 00000100:00100000:0.0:1789762862.035301:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067426688 00000100:00100000:0.0:1789762862.035336:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 272 00000100:00100000:0.0:1789762862.035342:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a12300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:14578:x1876702067426688:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:0.0:1789762862.035363:0:6418:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762862.035375:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6e00/0xdf5a7ac59f176072 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c847 expref: 33 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035400:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x13:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:0.0:1789762862.035406:0:6418:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762862.035415:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17614b lrc: 3/0,1 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035434:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17614b lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035440:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17614b lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035447:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f176152 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035453:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f176152 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035464:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f176152 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035501:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176159 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035506:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176159 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x13:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035511:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176159 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x13:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762862.035823:0:6418:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].11, 64 off8896 00000040:00080000:0.0:1789762862.035834:0:6418:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].11, 64 off8896 00010000:00010000:0.0:1789762862.035878:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f176160 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035886:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f176160 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035891:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f176160 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035895:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f176160 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035900:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f176160 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035904:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f176160 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035939:0:6418:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762862.035951:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176159 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x13:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035960:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176159 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x13:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035969:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176159 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x13:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.035969:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762862.035978:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f176160 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.035981:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762862.035989:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176159 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x13:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.035990:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762862.035996:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f176160 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.036007:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176159 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x13:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.036010:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f176160 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.036026:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x13:0x0].0x0 (ffff9c37e125af00) 00010000:00010000:0.0:1789762862.036031:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762862.036035:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176159 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x13:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.036040:0:8069:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x13:0x0].0x0 (ffff9c37e125af00) 00010000:00010000:0.0:1789762862.036044:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176159 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x13:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.036044:0:8069:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789762862.036050:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f176160 lrc: 1/0,0 mode: --/PW res: [0x240000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.036063:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f176160 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.036070:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17614b lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.036075:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762862.036078:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17614b lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.036079:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762862.036083:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762862.036088:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17614b lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.036106:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17614b lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.036111:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762862.036115:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17614b lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.036122:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17614b lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.036132:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762862.036135:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762862.036139:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17614b lrc: 3/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.036147:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769200/0xdf5a7ac59f17614b lrc: 0/0,0 mode: --/CW res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762862.036159:0:6418:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004436ec6a reply state 00000000f9ac09c5 transno 4294967420 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f176152 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.036167:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f176152 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.036172:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x2b0e463d (ffff9c37d3bee100) 00010000:00010000:0.0:1789762862.036174:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762862.036176:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762862.036210:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a12300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:14578:x1876702067426688:12345-192.168.203.37@tcp:36:rm.0 Request processed in 868us (955us total) trans 4294967420 rc 0/0 00000100:00100000:0.0:1789762862.036216:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 272 00000100:00100000:1.0:1789762862.037058:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762862.037096:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067426816 00000100:00100000:0.0:1789762862.037102:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067426816 00000100:00100000:0.0:1789762862.037148:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 52 00000100:00100000:0.0:1789762862.037161:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a12680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067426816:12345-192.168.203.37@tcp:103:rm.0 00010000:00010000:0.0:1789762862.037174:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762862.037192:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f175f99 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c7fa expref: 7 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762862.037194:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:0.0:1789762862.037209:0:9878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a200) 00010000:00010000:0.0:1789762862.037218:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762862.037224:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067427072 00000100:00100000:2.0:1789762862.037229:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067427072 00000100:00100000:0.0:1789762862.037254:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a12680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067426816:12345-192.168.203.37@tcp:103:rm.0 Request processed in 97us (195us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762862.037260:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 77 00000100:00100000:0.0:1789762862.037266:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 52 00000100:00100000:2.0:1789762862.037271:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:14578:x1876702067427072:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:2.0:1789762862.037389:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:14578:x1876702067427072:12345-192.168.203.37@tcp:35:rm.0 Request processed in 111us (188us total) trans 4294967421 rc 0/0 00000100:00100000:2.0:1789762862.037397:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 77 00000100:00100000:1.0:1789762862.038044:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.038071:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067427328 00000100:00100000:2.0:1789762862.038075:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067427328 00000100:00100000:2.0:1789762862.038098:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 273 00000100:00100000:2.0:1789762862.038105:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a13b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:14578:x1876702067427328:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:2.0:1789762862.038128:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1789762862.038137:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273ca00/0xdf5a7ac59f176121 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x11:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8b0 expref: 35 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.038150:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f17601e lrc: 0/0,0 mode: --/PR res: [0x240000402:0x11:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c82b expref: 34 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.038158:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:2.0:1789762862.038160:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762862.038194:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.038200:0:6419:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.038207:0:6419:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762862.038225:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a09f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074696192:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1789762862.038260:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762862.038295:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074696192 00000100:00100000:0.0:1789762862.038300:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074696192 00000100:00100000:2.0:1789762862.038301:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37eebc6840 going to sleep for 11 seconds 00000100:00100000:0.0:1789762862.038332:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 320 00000100:00100000:0.0:1789762862.038339:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:6419:x1876702074696192:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1789762862.038347:0:7227:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762862.038357:0:7227:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f17616e lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f176167 expref: -99 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.038373:0:7227:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175fae lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c808 expref: 5 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.038397:0:7227:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f17616e lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f176167 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.038419:0:7227:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175fae lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c808 expref: 5 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.038437:0:7227:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175fae lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c808 expref: 5 pid: 6418 timeout: 435 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762862.038446:0:7227:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:7227:1876702074696320:192.168.203.37@tcp:104: 00000100:00100000:0.0:1789762862.038489:0:7227:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f900 going to sleep for 11 seconds 00000100:00100000:0.0:1789762862.039187:0:7227:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:7227:1876702074696320:192.168.203.37@tcp:104: 00010000:00010000:0.0:1789762862.039210:0:7227:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f17616e lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176167 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.039217:0:7227:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762862.039220:0:7227:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762862.039223:0:7227:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f17616e lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176167 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.039229:0:7227:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769400, rc 0) 00000100:00100000:1.0:1789762862.039267:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762862.039271:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6419:x1876702074696192:12345-0@lo:101:mdt00_001.0 Request processed in 931us (1008us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762862.039278:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 320 00000100:00100000:2.0:1789762862.039291:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a09f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074696192:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1789762862.039306:0:6419:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.039314:0:6419:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762862.039314:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067427456 00000100:00100000:3.0:1789762862.039321:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067427456 00010000:00010000:3.0:1789762862.039347:0:8070:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175fae lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c808 expref: 6 pid: 6418 timeout: 435 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762862.039393:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 78 00000100:00100000:3.0:1789762862.039410:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c92450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067427456:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762862.039420:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762862.039427:0:8070:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175fae lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c808 expref: 6 pid: 6418 timeout: 435 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.039440:0:8070:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175fae lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c808 expref: 6 pid: 6418 timeout: 435 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.039455:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f175fae lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c808 expref: 6 pid: 6418 timeout: 435 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.039467:0:8070:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762862.039473:0:8070:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f17616e lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176167 expref: 6 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.039484:0:8070:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f17616e lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f176167 expref: 6 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.039493:0:8070:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f17616e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f176167 expref: 6 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.039530:0:8070:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f17616e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176167 expref: 6 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762862.039542:0:8070:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9f6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:8070:1876702074696448:0@lo:105: 00000100:00100000:3.0:1789762862.039573:0:8070:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762862.039618:0:8068:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074696448 00000100:00100000:0.0:1789762862.039622:0:8068:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074696448 00000100:00100000:3.0:1789762862.039622:0:8070:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252fa80 going to sleep for 11 seconds 00000100:00100000:0.0:1789762862.039656:0:8068:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 51 00000100:00100000:0.0:1789762862.039662:0:8068:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:8070:x1876702074696448:12345-0@lo:105: 00010000:00010000:0.0:1789762862.039671:0:8068:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.039688:0:8068:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.039696:0:8068:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.039701:0:8068:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.039706:0:8068:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.039712:0:8068:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.039725:0:8068:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c37c9bc7a00) 00010000:00010000:2.0:1789762862.039746:0:6419:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762862.039756:0:8068:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:8070:x1876702074696448:12345-0@lo:105: Request processed in 94us (183us total) trans 0 rc 0/0 00010000:00010000:2.0:1789762862.039756:0:6419:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762862.039773:0:8068:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 51 00000004:00080000:2.0:1789762862.039782:0:6419:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1789762862.039795:0:8070:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9f6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:8070:1876702074696448:0@lo:105: 00010000:00010000:2.0:1789762862.039805:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.039811:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762862.039812:0:6419:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.039820:0:6419:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762862.039830:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a09f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074696576:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1789762862.039856:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762862.039857:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c92450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067427456:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 446us (590us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762862.039868:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 78 00000100:00100000:2.0:1789762862.039896:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37eebc6840 going to sleep for 11 seconds 00000100:00100000:1.0:1789762862.039943:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074696576 00000100:00100000:1.0:1789762862.039949:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074696576 00000100:00100000:1.0:1789762862.039991:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 321 00000100:00100000:1.0:1789762862.040002:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfeed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6419:x1876702074696576:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1789762862.040021:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762862.040040:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998e600/0xdf5a7ac59f17617c lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f176175 expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.040055:0:6424:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998e600/0xdf5a7ac59f17617c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f176175 expref: 8 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.040064:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998e600/0xdf5a7ac59f17617c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176175 expref: 8 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.040074:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743300) 00010000:00010000:1.0:1789762862.040076:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762862.040078:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d998e600, rc 0) 00000100:00100000:1.0:1789762862.040117:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfeed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074696576:12345-0@lo:101:mdt00_001.0 Request processed in 118us (262us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762862.040123:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 321 00000100:00100000:2.0:1789762862.040134:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a09f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074696576:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1789762862.040145:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f17617c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.040152:0:6419:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17617c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762862.040160:0:6419:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1789762862.040191:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfefb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074696704:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762862.040224:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762862.040239:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37eebc6600 going to sleep for 11 seconds 00000100:00100000:1.0:1789762862.040256:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074696704 00000100:00100000:1.0:1789762862.040261:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074696704 00000100:00100000:1.0:1789762862.040283:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 322 00000100:00100000:1.0:1789762862.040288:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074696704:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1789762862.040349:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074696704:12345-0@lo:1000:mdt00_001.0 Request processed in 61us (124us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762862.040355:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 322 00000100:00100000:2.0:1789762862.040375:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfefb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074696704:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762862.040416:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfefb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074696832:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762862.040434:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762862.040448:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37eebc6600 going to sleep for 11 seconds 00000100:00100000:0.0:1789762862.040459:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074696832 00000100:00100000:0.0:1789762862.040464:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074696832 00000100:00100000:0.0:1789762862.040487:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 323 00000100:00100000:0.0:1789762862.040493:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfeca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074696832:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789762862.040617:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfeca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074696832:12345-0@lo:1000:mdt00_001.0 Request processed in 123us (180us total) trans 0 rc 1/1 00000100:00100000:0.0:1789762862.040623:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 323 00000100:00100000:2.0:1789762862.040639:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfefb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074696832:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1789762862.040667:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.040672:0:6419:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.040677:0:6419:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762862.040684:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfefb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074696960:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1789762862.040701:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762862.040714:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37eebc6840 going to sleep for 11 seconds 00000100:00100000:0.0:1789762862.040720:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074696960 00000100:00100000:0.0:1789762862.040725:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074696960 00000100:00100000:0.0:1789762862.040755:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 324 00000100:00100000:0.0:1789762862.040760:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6419:x1876702074696960:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1789762862.040767:0:7227:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762862.040777:0:7227:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f17618a lrc: 2/0,0 mode: --/EX res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f176183 expref: -99 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.040790:0:7227:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f17618a lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f176183 expref: 9 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.040797:0:7227:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f17618a lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176183 expref: 9 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.040803:0:7227:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125ab00) 00010000:00010000:0.0:1789762862.040805:0:7227:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762862.040807:0:7227:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769a00, rc 0) 00000100:00100000:0.0:1789762862.040832:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6419:x1876702074696960:12345-0@lo:101:mdt00_001.0 Request processed in 71us (130us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762862.040836:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 324 00000100:00100000:2.0:1789762862.040853:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfefb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6419:1876702074696960:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1789762862.040863:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f17618a expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.040873:0:6419:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17618a expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.040892:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176191 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.040908:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176191 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.040952:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176191 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789762862.041398:0:6419:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].10, 424 off39808 00000040:00080000:2.0:1789762862.041436:0:6419:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].10, 504 off40968 00000004:00080000:2.0:1789762862.041442:0:6419:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37eebc6600 with version 21 00000004:00080000:1.0:1789762862.041484:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 21 rpc_version 21 00000020:00100000:1.0:1789762862.041508:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1789762862.041517:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c36d2a13480 x1876702074697216/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1744/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762862.041530:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d2a13480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074697216:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1789762862.041559:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762862.041577:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c9a9d780 going to sleep for 11 seconds 00000100:00100000:3.0:1789762862.041601:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074697216 00000100:00100000:3.0:1789762862.041607:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074697216 00000100:00100000:3.0:1789762862.041641:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 325 00000100:00100000:3.0:1789762862.041650:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a11180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7224:x1876702074697216:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1789762862.042232:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a11180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7224:x1876702074697216:12345-0@lo:1000:osp_up0-1.0 Request processed in 582us (670us total) trans 4294967406 rc 0/0 00000100:00100000:3.0:1789762862.042242:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 325 00000100:00100000:1.0:1789762862.042275:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d2a13480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074697216:0@lo:1000:osp_up0-1.0 00000004:00200000:2.0:1789762862.042295:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x11:0x0]: returning size 16384 00010000:00010000:2.0:1789762862.042305:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176191 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042310:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176191 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042315:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176191 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042318:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762862.042320:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176191 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042324:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176191 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042330:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:2.0:1789762862.042332:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762862.042334:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176191 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042347:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176191 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042355:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17618a expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042360:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f17618a expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042365:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f17618a expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042392:0:6419:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1789762862.042398:0:6419:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1789762862.042402:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042407:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.042413:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762862.042414:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042436:0:6419:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1789762862.042439:0:6419:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1789762862.042442:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f17617c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.042446:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f17618a expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.042457:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762862.042466:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042473:0:6419:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f17617c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.042494:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7a00 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:1.0:1789762862.042495:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7e00 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:0.0:1789762862.042502:0:6407:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.042505:0:6408:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f17618a expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762862.042515:0:6407:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762862.042520:0:6408:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f17618a expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762862.042526:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a13b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:14578:x1876702067427328:12345-192.168.203.37@tcp:36:rm.0 Request processed in 4419us (4480us total) trans 4294967422 rc 0/0 00000100:00100000:2.0:1789762862.042545:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 273 00010000:00010000:0.0:1789762862.042569:0:6407:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.042572:0:6408:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f17618a expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.042583:0:6407:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789762862.042590:0:6408:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1789762862.042610:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762862.042626:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074697344:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762862.042629:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7a00/0xdf5a7ac59f176167 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f17616e expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.042631:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f17618a expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.042648:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7e00/0xdf5a7ac59f176183 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f17618a expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.042649:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762862.042653:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762862.042656:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762862.042658:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789762862.042678:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762862.042682:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762862.042687:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762862.042689:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074697344 00000100:00100000:0.0:1789762862.042693:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074697344 00000100:00100000:3.0:1789762862.042696:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789762862.042724:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d2a10a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074697472:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762862.042734:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 53 00000100:00100000:0.0:1789762862.042743:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3427:x1876702074697344:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762862.042751:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762862.042762:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f17616e lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f176167 expref: 10 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762862.042768:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1789762862.042778:0:6406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762862.042782:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762862.042790:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074697472 00000100:00100000:2.0:1789762862.042795:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074697472 00000100:00100000:0.0:1789762862.042825:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3427:x1876702074697344:12345-0@lo:103:ldlm_bl.0 Request processed in 82us (170us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762862.042826:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 54 00000100:00100000:0.0:1789762862.042840:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 53 00000100:00100000:2.0:1789762862.042841:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3425:x1876702074697472:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762862.042861:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1789762862.042863:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074697344:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762862.042877:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f17618a lrc: 0/0,0 mode: --/EX res: [0x240000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f176183 expref: 9 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.042891:0:6405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125ab00) 00010000:00010000:2.0:1789762862.042899:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762862.042963:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3425:x1876702074697472:12345-0@lo:103:ldlm_bl.0 Request processed in 126us (200us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762862.042969:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 54 00000100:00100000:3.0:1789762862.043000:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d2a10a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074697472:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1789762862.335289:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762862.335363:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067427584 00000100:00100000:0.0:1789762862.335404:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067427584 00000100:00100000:0.0:1789762862.335494:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 274 00000100:00100000:0.0:1789762862.335522:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a13100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:14589:x1876702067427584:12345-192.168.203.37@tcp:101:mkdir.0 00010000:00010000:0.0:1789762862.335547:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762862.335578:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f176198 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8b7 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.335654:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f17619f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.335669:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f17619f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.335681:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f17619f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762862.335737:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762862.335740:0:11692:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762862.335759:0:11692:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f17619f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.335800:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f176198 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8b7 expref: 7 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.335814:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f17619f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8b7 expref: 6 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.335822:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762862.335825:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762862.335828:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769c00, rc 0) 00000100:00100000:0.0:1789762862.335897:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a13100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:14589:x1876702067427584:12345-192.168.203.37@tcp:101:mkdir.0 Request processed in 395us (634us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762862.335903:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 274 00000100:00100000:1.0:1789762862.337520:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.337586:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067427712 00000100:00100000:1.0:1789762862.337589:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.337593:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067427712 00000100:00100000:2.0:1789762862.337644:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067427968 00000100:00100000:2.0:1789762862.337649:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067427968 00000100:00100000:0.0:1789762862.337651:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 275 00000100:00100000:0.0:1789762862.337661:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a10380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067427712:12345-192.168.203.37@tcp:41:mkdir.0 00000100:00100000:2.0:1789762862.337684:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 276 00000004:00000024:0.0:1789762862.337703:0:6418:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762862.337704:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a12a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14589:x1876702067427968:12345-192.168.203.37@tcp:36:mkdir.0 00010000:00010000:2.0:1789762862.337744:0:6420:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1789762862.337756:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a10380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067427712:12345-192.168.203.37@tcp:41:mkdir.0 Request processed in 96us (240us total) trans 0 rc 0/0 00010000:00010000:2.0:1789762862.337763:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f17619f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8b7 expref: 7 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762862.337764:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 275 00010000:00010000:2.0:1789762862.337778:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762862.337782:0:6420:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762862.337856:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761a6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.337865:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761a6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.337872:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761a6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.337879:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.337888:0:6420:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998e600/0xdf5a7ac59f17617c lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176175 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.337895:0:6420:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 4/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.337990:0:6420:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998e600/0xdf5a7ac59f17617c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xdf5a7ac59f176175 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762862.338007:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8ab480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6420:1876702074697600:0@lo:104: 00000100:00100000:2.0:1789762862.338055:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762862.338100:0:6403:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074697600 00000100:00100000:0.0:1789762862.338105:0:6403:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074697600 00000100:00100000:2.0:1789762862.338121:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f900 going to sleep for 11 seconds 00000100:00100000:0.0:1789762862.338143:0:6403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:0.0:1789762862.338154:0:6403:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8abb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6420:x1876702074697600:12345-0@lo:104: 00010000:00010000:0.0:1789762862.338178:0:6403:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f17617c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.338235:0:6403:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1789762862.338243:0:6403:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8abb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6420:x1876702074697600:12345-0@lo:104: Request processed in 92us (186us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762862.338251:0:6403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00010000:00010000:1.0:1789762862.338262:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:2.0:1789762862.338263:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8ab480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6420:1876702074697600:0@lo:104: 00010000:00010000:1.0:1789762862.338277:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f17617c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.338291:0:6420:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.338304:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7600 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:1.0:1789762862.338313:0:7217:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xdf5a7ac59f17617c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762862.338326:0:7217:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xdf5a7ac59f17617c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.338358:0:7217:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f17617c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.338367:0:7217:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789762862.338417:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f17617c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.338427:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c9bc7600/0xdf5a7ac59f176175 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f17617c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762862.338439:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d2a10e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074697728:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762862.338452:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762862.338457:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762862.338462:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762862.338483:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762862.338517:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074697728 00000100:00100000:0.0:1789762862.338521:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074697728 00010000:00010000:0.0:1789762862.338540:0:6404:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998e600/0xdf5a7ac59f17617c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xdf5a7ac59f176175 expref: 8 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762862.338566:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 79 00000100:00100000:0.0:1789762862.338572:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8ab480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3425:x1876702074697728:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762862.338587:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1789762862.348705:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967406 is committed 00010000:00010000:1.0:1789762862.350608:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998e600/0xdf5a7ac59f17617c lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xdf5a7ac59f176175 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.350628:0:6404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743300) 00010000:00010000:1.0:1789762862.350635:0:6404:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.350648:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.350659:0:6404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.350710:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762862.350734:0:6420:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.350763:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762862.350832:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8ab480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074697728:12345-0@lo:103:ldlm_bl.0 Request processed in 12256us (12346us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762862.350846:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 79 00000100:00100000:3.0:1789762862.350892:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d2a10e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074697728:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762862.352336:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1761b4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352356:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1761b4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352367:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1761b4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762862.352403:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f081620b reply state 0000000052a56a17 transno 4294967407 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1761b4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352412:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1761b4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352418:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762862.352421:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762862.352424:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762862.352444:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 16384 00010000:00010000:2.0:1789762862.352454:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761a6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352460:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761a6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352466:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761a6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352471:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761a6 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352476:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762862.352479:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761a6 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352484:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761a6 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352494:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762862.352495:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762862.352498:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761a6 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352503:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761a6 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762862.352518:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f081620b reply state 0000000052a56a17 transno 4294967407 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352524:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.352537:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743300) 00010000:00010000:2.0:1789762862.352539:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762862.352541:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:2.0:1789762862.352661:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a12a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14589:x1876702067427968:12345-192.168.203.37@tcp:36:mkdir.0 Request processed in 14958us (15071us total) trans 4294967407 rc 0/0 00000100:00100000:2.0:1789762862.352672:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 276 00000100:00100000:1.0:1789762862.359294:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.359367:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067428096 00000100:00100000:2.0:1789762862.359391:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067428096 00000100:00100000:2.0:1789762862.359441:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 277 00000100:00100000:2.0:1789762862.359453:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a11500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14590:x1876702067428096:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762862.359472:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762862.359485:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761bb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8be expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.359531:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7200/0xdf5a7ac59f1761c2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.359540:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7200/0xdf5a7ac59f1761c2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.359548:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7200/0xdf5a7ac59f1761c2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762862.359573:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762862.359576:0:7744:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762862.359589:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7200/0xdf5a7ac59f1761c2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.359608:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f1761bb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8be expref: 9 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.359632:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7200/0xdf5a7ac59f1761c2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8be expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.359648:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762862.359650:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762862.359652:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c9bc7200, rc 0) 00000100:00100000:2.0:1789762862.359690:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a11500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:14590:x1876702067428096:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 242us (402us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762862.359695:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 277 00000100:00100000:1.0:1789762862.360841:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762862.360879:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067428352 00000100:00100000:3.0:1789762862.360886:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067428352 00000100:00100000:3.0:1789762862.360952:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 278 00000100:00100000:3.0:1789762862.360965:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a11880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:14590:x1876702067428352:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:3.0:1789762862.360981:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762862.360984:0:13389:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1789762862.361003:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7200/0xdf5a7ac59f1761c2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8be expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361036:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762862.361040:0:13389:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762862.361052:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d800/0xdf5a7ac59f1761c9 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8c5 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361164:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f1761d0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361175:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f1761d0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361201:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f1761d0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361217:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f1761d7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361227:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f1761d7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361237:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f1761d7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361337:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f1761de lrc: 3/1,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361347:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f1761de lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361353:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f1761de lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762862.361368:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 16384 00010000:00010000:3.0:1789762862.361397:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f1761de lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361404:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f1761de lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361419:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f1761de lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361423:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762862.361443:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f1761de lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361448:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f1761de lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361455:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:3.0:1789762862.361457:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762862.361460:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f1761de lrc: 2/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361464:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f1761de lrc: 0/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361478:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f1761d0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361482:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f1761d0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361494:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f1761d0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361498:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f1761d0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361501:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762862.361504:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f1761d0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361508:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f1761d0 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361513:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762862.361515:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762862.361517:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f1761d0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361521:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f1761d0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361527:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f1761d7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361531:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f1761d7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361535:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f1761d7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361539:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762862.361546:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f1761d7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361552:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f1761d7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361560:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743300) 00010000:00010000:3.0:1789762862.361563:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762862.361565:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f1761d7 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361569:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f1761d7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361599:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d800/0xdf5a7ac59f1761c9 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8c5 expref: 8 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361607:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762862.361609:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762862.361615:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d800/0xdf5a7ac59f1761c9 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8c5 expref: 8 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.361639:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d273d800, rc 0) 00000100:00100000:3.0:1789762862.361695:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a11880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14590:x1876702067428352:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 733us (856us total) trans 0 rc 301/301 00000100:00100000:3.0:1789762862.361701:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 278 00000100:00100000:1.0:1789762862.362650:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762862.362699:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067428480 00000100:00100000:0.0:1789762862.362705:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067428480 00000100:00100000:0.0:1789762862.362791:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 279 00000100:00100000:0.0:1789762862.362807:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a11c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:14590:x1876702067428480:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762862.362816:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762862.362827:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f1761e5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8cc expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.362863:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f1761ec lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.362881:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f1761ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.362888:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f1761ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762862.362904:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762862.362907:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762862.362919:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f1761ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.362976:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f1761e5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8cc expref: 9 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.363001:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f1761ec lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8cc expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.363018:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762862.363021:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762862.363024:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768c00, rc 0) 00000100:00100000:0.0:1789762862.363054:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a11c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:14590:x1876702067428480:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 255us (409us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762862.363060:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 279 00000100:00100000:3.0:1789762862.364074:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762862.364106:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067428736 00000100:00100000:0.0:1789762862.364111:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067428736 00000100:00100000:0.0:1789762862.364140:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 280 00000100:00100000:0.0:1789762862.364147:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:14590:x1876702067428736:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762862.364155:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762862.364189:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f1761f3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8d3 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364228:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f1761fa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364237:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f1761fa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364243:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f1761fa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364252:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f176201 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364257:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f176201 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364273:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f176201 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762862.364333:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 16384 00010000:00010000:0.0:1789762862.364460:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f1761fa lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364466:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f1761fa lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364470:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f1761fa lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364474:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f1761fa lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364478:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762862.364481:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f1761fa lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364485:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f1761fa lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364492:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762862.364495:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762862.364498:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f1761fa lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364503:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f1761fa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364520:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f176201 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364525:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f176201 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364529:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f176201 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364532:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762862.364534:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f176201 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364538:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f176201 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364542:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743300) 00010000:00010000:0.0:1789762862.364544:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762862.364546:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f176201 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364549:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f176201 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364567:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f1761f3 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8d3 expref: 10 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364573:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762862.364575:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762862.364583:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f1761f3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8d3 expref: 10 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.364590:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769800, rc 0) 00000100:00100000:0.0:1789762862.364624:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:14590:x1876702067428736:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 478us (551us total) trans 4294967408 rc 301/301 00000100:00100000:0.0:1789762862.364629:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 280 00000100:00100000:0.0:1789762862.365587:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.365631:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067428992 00000100:00100000:2.0:1789762862.365637:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067428992 00000100:00100000:2.0:1789762862.365672:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 281 00000100:00100000:2.0:1789762862.365684:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:14590:x1876702067428992:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:2.0:1789762862.365730:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f17620f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.365739:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f17620f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.365746:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f17620f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762862.365902:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002f3cfe33 reply state 000000005d0c3342 transno 4294967409 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f17620f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.365912:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f17620f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.365973:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762862.365975:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:2.0:1789762862.365978:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762862.365985:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 16384 00000100:00100000:2.0:1789762862.366030:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:14590:x1876702067428992:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 351us (443us total) trans 4294967409 rc 0/0 00000100:00100000:2.0:1789762862.366038:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 281 00000100:00100000:2.0:1789762862.366715:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762862.366758:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067429248 00000100:00100000:0.0:1789762862.366765:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067429248 00000100:00100000:0.0:1789762862.366803:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 78 00000100:00100000:0.0:1789762862.366813:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:14590:x1876702067429248:12345-192.168.203.37@tcp:35:lfs.0 00000100:00100000:0.0:1789762862.366965:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:14590:x1876702067429248:12345-192.168.203.37@tcp:35:lfs.0 Request processed in 155us (251us total) trans 4294967410 rc 0/0 00000100:00100000:0.0:1789762862.366974:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 78 00000100:00100000:3.0:1789762862.370816:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762862.370857:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067429376 00000100:00100000:3.0:1789762862.370865:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067429376 00000100:00100000:3.0:1789762862.370984:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 282 00000100:00100000:3.0:1789762862.370997:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:14591:x1876702067429376:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:3.0:1789762862.371021:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762862.371049:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d800/0xdf5a7ac59f176216 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8da expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371122:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17621d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371139:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17621d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371153:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17621d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371170:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176224 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371179:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176224 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371190:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176224 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371311:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f17622b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371321:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f17622b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371330:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f17622b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762862.371389:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 16384 00000004:00200000:3.0:1789762862.371394:0:6420:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762862.371414:0:6420:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f17622b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371429:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17621d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371438:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17621d lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371447:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17621d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371455:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17621d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371463:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762862.371469:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17621d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371480:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17621d lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371495:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762862.371500:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762862.371505:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17621d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371513:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f17621d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371549:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176224 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371557:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176224 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371565:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176224 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371572:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762862.371576:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176224 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371585:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176224 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371596:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743300) 00010000:00010000:3.0:1789762862.371600:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762862.371605:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176224 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371612:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176224 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371655:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d800/0xdf5a7ac59f176216 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8da expref: 14 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371683:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f17622b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8da expref: 13 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.371716:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:3.0:1789762862.371720:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762862.371724:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d273c200, rc 0) 00000100:00100000:3.0:1789762862.371780:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:14591:x1876702067429376:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 786us (969us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762862.371790:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 282 00000100:00100000:0.0:1789762862.372806:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.372845:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067429632 00000100:00100000:2.0:1789762862.372852:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067429632 00000100:00100000:2.0:1789762862.372890:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 283 00000100:00100000:2.0:1789762862.372899:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:14591:x1876702067429632:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:2.0:1789762862.372968:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762862.372988:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c200/0xdf5a7ac59f17622b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8da expref: 13 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.373003:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762862.373007:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762862.373019:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176232 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.373027:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176232 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.373035:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176232 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762862.373161:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b21b892d reply state 00000000105ad176 transno 4294967411 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176232 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.373177:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176232 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.373182:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762862.373184:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:2.0:1789762862.373186:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762862.373192:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 16384 00000100:00100000:2.0:1789762862.373250:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14591:x1876702067429632:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 351us (443us total) trans 4294967411 rc 0/0 00000100:00100000:2.0:1789762862.373260:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 283 00000100:00100000:0.0:1789762862.391366:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762862.391432:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067429760 00000100:00100000:3.0:1789762862.391443:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067429760 00000100:00100000:3.0:1789762862.391496:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 284 00000100:00100000:3.0:1789762862.391506:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14597:x1876702067429760:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762862.391526:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762862.391541:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d800/0xdf5a7ac59f176239 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8e1 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391593:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176240 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391605:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176240 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391614:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176240 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391622:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f176247 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391627:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f176247 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391644:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f176247 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391730:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273da00/0xdf5a7ac59f17624e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391735:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273da00/0xdf5a7ac59f17624e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391741:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273da00/0xdf5a7ac59f17624e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762862.391756:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 16384 00000004:00200000:3.0:1789762862.391758:0:13389:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762862.391773:0:13389:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273da00/0xdf5a7ac59f17624e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391795:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176240 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391801:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176240 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391807:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176240 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391811:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176240 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391815:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762862.391818:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176240 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391830:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176240 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391839:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762862.391842:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762862.391844:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176240 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391849:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273c400/0xdf5a7ac59f176240 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391859:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f176247 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391863:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f176247 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391868:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f176247 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391871:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762862.391873:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f176247 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391877:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f176247 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391881:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743300) 00010000:00010000:3.0:1789762862.391882:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762862.391884:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f176247 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391894:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d400/0xdf5a7ac59f176247 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391914:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d800/0xdf5a7ac59f176239 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8e1 expref: 16 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391954:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273da00/0xdf5a7ac59f17624e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8e1 expref: 15 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.391961:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:3.0:1789762862.391963:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789762862.391965:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d273da00, rc 0) 00000100:00100000:3.0:1789762862.392011:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:14597:x1876702067429760:12345-192.168.203.37@tcp:101:dd.500 Request processed in 508us (652us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762862.392018:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 284 00000100:00100000:1.0:1789762862.393310:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762862.393368:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067430016 00000100:00100000:0.0:1789762862.393394:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067430016 00000100:00100000:0.0:1789762862.393439:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 285 00000100:00100000:0.0:1789762862.393449:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a12d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:14597:x1876702067430016:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:0.0:1789762862.393460:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762862.393471:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f176255 lrc: 2/0,0 mode: --/CW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8e8 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.393563:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17625c lrc: 3/0,1 mode: --/CW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.393586:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17625c lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.393592:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17625c lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.393605:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f176263 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.393613:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f176263 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.393621:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f176263 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762862.394295:0:6418:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x7:0x0], index 0: 0 00010000:00010000:0.0:1789762862.394494:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f17626a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394505:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f17626a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394514:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f17626a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394553:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17625c lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394561:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17625c lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394568:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17625c lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394582:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17625c lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394589:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762862.394593:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17625c lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394601:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17625c lrc: 5/0,0 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394611:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:0.0:1789762862.394614:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762862.394618:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17625c lrc: 3/0,0 mode: --/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394626:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17625c lrc: 0/0,0 mode: --/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762862.394638:0:6418:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000019f2d7a3 reply state 000000003ad80404 transno 4294967412 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f176263 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394650:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f176263 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394657:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x4009f790 (ffff9c37e125ab00) 00010000:00010000:0.0:1789762862.394659:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762862.394662:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762862.394686:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f176255 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8e8 expref: 18 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394700:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f17626a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8e8 expref: 17 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.394716:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff9c37e125a200) 00010000:00010000:0.0:1789762862.394718:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762862.394721:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769a00, rc 0) 00000100:00100000:0.0:1789762862.394762:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a12d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:14597:x1876702067430016:12345-192.168.203.37@tcp:101:dd.500 Request processed in 1315us (1453us total) trans 4294967412 rc 0/0 00000100:00100000:0.0:1789762862.394770:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 285 00000100:00100000:0.0:1789762862.398035:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762862.398077:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067430144 00000100:00100000:3.0:1789762862.398082:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067430144 00000100:00100000:3.0:1789762862.398125:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335134834387 time 335135046218 state 0x0 ref 1 00000100:00100000:3.0:1789762862.398137:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335185046218 time 335135046218 state 0x1 ref 1 00000100:00100000:3.0:1789762862.398160:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 2 deadline 335185046218 time 335645237942 state 0x0 ref 1 00000100:00100000:3.0:1789762862.398170:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2528700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2395:x1876702067430144:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762862.398206:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762862.398217:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f176278 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.398230:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=335s v=(0 0) 00010000:00010000:3.0:1789762862.398234:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762862.398239:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f176278 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.398249:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f176278 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762862.421462:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967807 is committed 00002000:00010000:3.0:1789762862.421505:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:3.0:1789762862.421511:0:13301:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb mtime from disk: 0 -> 1789762862 00002000:00010000:3.0:1789762862.421513:0:13301:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb ctime from disk: 0 -> 1789762862 00002000:00010000:3.0:1789762862.421515:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 1 -> 2051 00000001:00080000:0.0:1789762862.421522:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967808 is committed 00010000:00010000:3.0:1789762862.421532:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f176278 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.421546:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f176278 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762862.421550:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967810 is committed 00010000:00010000:3.0:1789762862.421556:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f176278 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.421563:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762862.421568:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f176278 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.421588:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f176278 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.421608:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f176278 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.421613:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f176278 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762862.421715:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2528700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067430144:12345-192.168.203.37@tcp:4:dd.500 Request processed in 23539us (23675us total) trans 4294967810 rc 0/0 00000100:00100000:3.0:1789762862.421725:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 493 00000100:00100000:3.0:1789762862.421738:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 2 deadline 335185046218 time 335645237942 state 0x2 ref 0 00000100:00100000:1.0:1789762862.435436:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762862.435490:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067430272 00000100:00100000:3.0:1789762862.435502:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067430272 00000100:00100000:3.0:1789762862.435570:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 2 deadline 335185046218 time 335645237942 state 0x0 ref 1 00000100:00100000:3.0:1789762862.435584:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 2 deadline 335695237942 time 335645237942 state 0x1 ref 1 00000100:00100000:3.0:1789762862.435612:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 1 deadline 335695237942 time 335682689144 state 0x0 ref 1 00000100:00100000:3.0:1789762862.435622:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067430272:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762862.435671:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 1048576, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x803 00010000:00010000:3.0:1789762862.435700:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17627f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.435721:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=335s v=(0 0) 00010000:00010000:3.0:1789762862.435727:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762862.435735:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17627f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.435750:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17627f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762862.448448:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967811 is committed 00002000:00010000:3.0:1789762862.448458:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:3.0:1789762862.448463:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:3.0:1789762862.448475:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17627f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.448487:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17627f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.448494:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17627f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.448499:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762862.448503:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17627f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.448511:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17627f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.448537:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17627f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.448543:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17627f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762862.448611:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a10700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067430272:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12986us (13186us total) trans 4294967811 rc 0/0 00000100:00100000:3.0:1789762862.448621:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 494 00000100:00100000:3.0:1789762862.448631:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 1 deadline 335695237942 time 335682689144 state 0x2 ref 0 00000100:00100000:0.0:1789762862.450974:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762862.451025:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067430400 00000100:00100000:1.0:1789762862.451033:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067430400 00000100:00100000:1.0:1789762862.451080:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 1 deadline 335695237942 time 335682689144 state 0x0 ref 1 00000100:00100000:1.0:1789762862.451097:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 1 deadline 335732689144 time 335682689144 state 0x1 ref 1 00000100:00100000:1.0:1789762862.451127:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335732689144 time 335698203699 state 0x0 ref 1 00000100:00100000:1.0:1789762862.451138:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067430400:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762862.451169:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 2097152, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x1007 00010000:00010000:1.0:1789762862.451175:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c643b83/0xdf5a7ac59f176286 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.451186:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=335s v=(0 0) 00010000:00010000:1.0:1789762862.451191:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762862.451196:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c643b83/0xdf5a7ac59f176286 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.451206:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c643b83/0xdf5a7ac59f176286 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762862.463483:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967812 is committed 00002000:00010000:3.0:1789762862.463507:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:3.0:1789762862.463511:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:3.0:1789762862.463520:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c643b83/0xdf5a7ac59f176286 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.463530:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c643b83/0xdf5a7ac59f176286 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.463536:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c643b83/0xdf5a7ac59f176286 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.463541:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762862.463544:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c643b83/0xdf5a7ac59f176286 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.463562:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c643b83/0xdf5a7ac59f176286 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.463574:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c643b83/0xdf5a7ac59f176286 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.463579:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c643b83/0xdf5a7ac59f176286 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762862.463632:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067430400:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12492us (12656us total) trans 4294967812 rc 0/0 00000100:00100000:3.0:1789762862.463642:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 495 00000100:00100000:3.0:1789762862.463650:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335732689144 time 335698203699 state 0x2 ref 0 00000100:00100000:0.0:1789762862.465824:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.465909:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067430528 00000100:00100000:2.0:1789762862.465952:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067430528 00000100:00100000:2.0:1789762862.466005:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335732689144 time 335698203699 state 0x0 ref 1 00000100:00100000:2.0:1789762862.466017:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335748203699 time 335698203699 state 0x1 ref 1 00000100:00100000:3.0:1789762862.501230:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335748203699 time 335748295860 state 0x0 ref 1 00000100:00100000:3.0:1789762862.501252:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067430528:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762862.501303:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 3145728, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x1807 00010000:00010000:3.0:1789762862.501310:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f17628d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.501325:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=335s v=(0 0) 00010000:00010000:3.0:1789762862.501328:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762862.501334:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f17628d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.501358:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f17628d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762862.514246:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967813 is committed 00002000:00010000:0.0:1789762862.514247:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:0.0:1789762862.514257:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:0.0:1789762862.514271:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f17628d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.514283:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f17628d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.514291:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f17628d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.514297:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762862.514308:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f17628d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.514316:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f17628d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.514337:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f17628d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.514344:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f17628d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762862.514438:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067430528:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13189us (48610us total) trans 4294967813 rc 0/0 00000100:00100000:0.0:1789762862.514450:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 496 00000100:00100000:0.0:1789762862.514461:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335748203699 time 335748295860 state 0x2 ref 0 00000100:00100000:1.0:1789762862.516883:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762862.516948:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067430656 00000100:00100000:0.0:1789762862.516959:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067430656 00000100:00100000:0.0:1789762862.517007:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335748203699 time 335748295860 state 0x0 ref 1 00000100:00100000:0.0:1789762862.517020:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335798295860 time 335748295860 state 0x1 ref 1 00000100:00100000:0.0:1789762862.551307:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335798295860 time 335798371559 state 0x0 ref 1 00000100:00100000:0.0:1789762862.551337:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a13800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067430656:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762862.551422:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 4194304, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x2007 00010000:00010000:0.0:1789762862.551430:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176294 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.551445:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=335s v=(0 0) 00010000:00010000:0.0:1789762862.551450:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762862.551456:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.551469:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762862.564084:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967814 is committed 00002000:00010000:2.0:1789762862.564134:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:2.0:1789762862.564137:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:2.0:1789762862.564149:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.564159:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176294 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.564166:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176294 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.564178:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762862.564182:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176294 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.564188:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176294 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.564202:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176294 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.564207:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176294 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762862.564282:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a13800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067430656:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12950us (47398us total) trans 4294967814 rc 0/0 00000100:00100000:2.0:1789762862.564291:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 497 00000100:00100000:2.0:1789762862.564301:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335798295860 time 335798371559 state 0x2 ref 0 00000100:00100000:1.0:1789762862.566706:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762862.566757:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067430784 00000100:00100000:3.0:1789762862.566767:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067430784 00000100:00100000:3.0:1789762862.566822:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335798295860 time 335798371559 state 0x0 ref 1 00000100:00100000:3.0:1789762862.566835:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335848371559 time 335798371559 state 0x1 ref 1 00000100:00100000:0.0:1789762862.592065:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074697856:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789762862.592147:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762862.592207:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074697856 00000100:00100000:1.0:1789762862.592217:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074697856 00000100:00100000:1.0:1789762862.592283:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 326 00000100:00100000:1.0:1789762862.592296:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074697856:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789762862.592336:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789762862.592412:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074697856:12345-0@lo:41:osp-pre-0-1.0 Request processed in 119us (264us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762862.592419:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 326 00000100:00100000:0.0:1789762862.592462:0:3426:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967406 gen 2 00000004:00080000:0.0:1789762862.592470:0:3426:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967406 committed transno 4294967406 00000100:00100000:0.0:1789762862.592505:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074697856:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789762862.601430:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335848371559 time 335848499990 state 0x0 ref 1 00000100:00100000:0.0:1789762862.601446:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a10e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067430784:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762862.601494:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 5242880, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x2807 00010000:00010000:0.0:1789762862.601502:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f17629b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.601519:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=335s v=(0 0) 00010000:00010000:0.0:1789762862.601535:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762862.601542:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f17629b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.601555:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f17629b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762862.614064:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967815 is committed 00002000:00010000:2.0:1789762862.614146:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:2.0:1789762862.614154:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:2.0:1789762862.614165:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f17629b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.614176:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f17629b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.614202:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f17629b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.614211:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762862.614216:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f17629b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.614226:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f17629b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.614244:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f17629b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.614249:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f17629b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762862.614326:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a10e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067430784:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12880us (47621us total) trans 4294967815 rc 0/0 00000100:00100000:2.0:1789762862.614335:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 498 00000100:00100000:2.0:1789762862.614345:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335848371559 time 335848499990 state 0x2 ref 0 00000100:00100000:1.0:1789762862.616780:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762862.616860:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067430912 00000100:00100000:0.0:1789762862.616871:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067430912 00000100:00100000:0.0:1789762862.616947:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335848371559 time 335848499990 state 0x0 ref 1 00000100:00100000:0.0:1789762862.616961:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335898499990 time 335848499990 state 0x1 ref 1 00000100:00100000:3.0:1789762862.651533:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335898499990 time 335898592326 state 0x0 ref 1 00000100:00100000:3.0:1789762862.651564:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a10a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067430912:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762862.651642:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 6291456, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x3007 00010000:00010000:3.0:1789762862.651651:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f1762a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.651667:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=335s v=(0 0) 00010000:00010000:3.0:1789762862.651677:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762862.651684:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f1762a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.651697:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f1762a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762862.664646:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:1.0:1789762862.664654:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 12295 -> 14343 00000001:00080000:2.0:1789762862.664654:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967816 is committed 00010000:00010000:1.0:1789762862.664668:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f1762a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.664680:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f1762a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.664687:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f1762a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.664698:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762862.664702:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f1762a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.664710:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f1762a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.664729:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f1762a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.664737:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f1762a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762862.664817:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a10a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067430912:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13263us (48037us total) trans 4294967816 rc 0/0 00000100:00100000:1.0:1789762862.664827:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 499 00000100:00100000:1.0:1789762862.664837:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335898499990 time 335898592326 state 0x2 ref 0 00000100:00100000:0.0:1789762862.667308:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.667359:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067431040 00000100:00100000:2.0:1789762862.667370:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067431040 00000100:00100000:2.0:1789762862.667436:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335898499990 time 335898592326 state 0x0 ref 1 00000100:00100000:2.0:1789762862.667462:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335948592326 time 335898592326 state 0x1 ref 1 00000100:00100000:2.0:1789762862.701583:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335948592326 time 335948647112 state 0x0 ref 1 00000100:00100000:2.0:1789762862.701608:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067431040:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762862.701673:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 7340032, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x3807 00010000:00010000:2.0:1789762862.701682:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1762a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.701698:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=335s v=(0 0) 00010000:00010000:2.0:1789762862.701702:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762862.701707:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1762a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.701721:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1762a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762862.714973:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967817 is committed 00002000:00010000:2.0:1789762862.714994:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:2.0:1789762862.715000:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:2.0:1789762862.715013:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1762a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.715024:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1762a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.715043:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1762a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.715049:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762862.715053:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1762a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.715060:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1762a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.715079:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1762a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.715085:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1762a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762862.715165:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067431040:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13562us (47858us total) trans 4294967817 rc 0/0 00000100:00100000:2.0:1789762862.715174:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 500 00000100:00100000:2.0:1789762862.715185:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335948592326 time 335948647112 state 0x2 ref 0 00000100:00100000:3.0:1789762862.717565:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.717602:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067431168 00000100:00100000:2.0:1789762862.717612:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067431168 00000100:00100000:2.0:1789762862.717659:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335948592326 time 335948647112 state 0x0 ref 1 00000100:00100000:2.0:1789762862.717680:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335998647112 time 335948647112 state 0x1 ref 1 00010000:00010000:1.0:1789762862.720046:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762862.751688:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335998647112 time 335998739204 state 0x0 ref 1 00000100:00100000:3.0:1789762862.751725:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067431168:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762862.751793:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 8388608, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x4007 00010000:00010000:3.0:1789762862.751802:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f1762b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.751820:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=335s v=(0 0) 00010000:00010000:3.0:1789762862.751824:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762862.751829:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f1762b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.751843:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f1762b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762862.764829:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967818 is committed 00002000:00010000:2.0:1789762862.764887:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 8388608 -> 9437184 00002000:00010000:2.0:1789762862.764892:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 16391 -> 18439 00010000:00010000:2.0:1789762862.764904:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f1762b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.764915:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f1762b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.764943:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f1762b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.764949:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762862.764953:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f1762b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.764961:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f1762b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.764981:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f1762b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.764987:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f1762b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762862.765082:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067431168:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13371us (47513us total) trans 4294967818 rc 0/0 00000100:00100000:2.0:1789762862.765096:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 501 00000100:00100000:2.0:1789762862.765109:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335998647112 time 335998739204 state 0x2 ref 0 00000100:00100000:0.0:1789762862.767559:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762862.767625:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067431296 00000100:00100000:1.0:1789762862.767638:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067431296 00000100:00100000:1.0:1789762862.767692:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 335998647112 time 335998739204 state 0x0 ref 1 00000100:00100000:1.0:1789762862.767707:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336048739204 time 335998739204 state 0x1 ref 1 00000100:00100000:3.0:1789762862.801757:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336048739204 time 336048817052 state 0x0 ref 1 00000100:00100000:3.0:1789762862.801788:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067431296:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762862.801848:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 9437184, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x4807 00010000:00010000:3.0:1789762862.801856:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f1762b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.801893:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:3.0:1789762862.801897:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762862.801903:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f1762b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762862.801937:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f1762b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762862.815311:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967819 is committed 00002000:00010000:1.0:1789762862.815322:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 9437184 -> 10485760 00002000:00010000:1.0:1789762862.815328:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 18439 -> 20487 00010000:00010000:1.0:1789762862.815343:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f1762b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.815355:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f1762b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.815362:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f1762b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.815367:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273dc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762862.815371:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f1762b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.815394:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f1762b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.815414:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f1762b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.815420:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f1762b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762862.815512:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067431296:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13731us (47955us total) trans 4294967819 rc 0/0 00000100:00100000:1.0:1789762862.815528:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 502 00000100:00100000:1.0:1789762862.815541:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336048739204 time 336048817052 state 0x2 ref 0 00000100:00100000:0.0:1789762862.818170:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.818229:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067431424 00000100:00100000:2.0:1789762862.818240:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067431424 00000100:00100000:2.0:1789762862.818290:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336048739204 time 336048817052 state 0x0 ref 1 00000100:00100000:2.0:1789762862.818304:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336098817052 time 336048817052 state 0x1 ref 1 00000100:00100000:0.0:1789762862.851851:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336098817052 time 336098916648 state 0x0 ref 1 00000100:00100000:0.0:1789762862.851878:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067431424:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762862.851950:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 10485760, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x5007 00010000:00010000:0.0:1789762862.851960:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1762be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.851977:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:0.0:1789762862.851998:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762862.852005:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1762be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.852019:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1762be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762862.865340:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967820 is committed 00002000:00010000:0.0:1789762862.865347:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:0.0:1789762862.865353:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:0.0:1789762862.865368:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1762be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.865394:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1762be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.865405:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1762be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.865414:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762862.865420:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1762be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.865431:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1762be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.865447:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1762be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.865464:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1762be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762862.865540:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067431424:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13671us (47372us total) trans 4294967820 rc 0/0 00000100:00100000:0.0:1789762862.865549:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 503 00000100:00100000:0.0:1789762862.865559:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336098817052 time 336098916648 state 0x2 ref 0 00000100:00100000:1.0:1789762862.868026:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762862.868079:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067431552 00000100:00100000:2.0:1789762862.868090:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067431552 00000100:00100000:2.0:1789762862.868147:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336098817052 time 336098916648 state 0x0 ref 1 00000100:00100000:2.0:1789762862.868161:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336148916648 time 336098916648 state 0x1 ref 1 00000100:00100000:2.0:1789762862.901969:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336148916648 time 336149025912 state 0x0 ref 1 00000100:00100000:2.0:1789762862.902010:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a12680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067431552:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762862.902081:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 11534336, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x5807 00010000:00010000:2.0:1789762862.902090:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1762c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.902130:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:2.0:1789762862.902135:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762862.902144:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1762c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.902161:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1762c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762862.913598:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967821 is committed 00002000:00010000:2.0:1789762862.913661:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 11534336 -> 12582912 00002000:00010000:2.0:1789762862.913667:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 22535 -> 24583 00010000:00010000:2.0:1789762862.913679:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1762c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.913689:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1762c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.913696:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1762c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.913702:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762862.913706:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1762c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.913714:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1762c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.913741:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1762c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762862.913747:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1762c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762862.913815:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a12680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067431552:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11826us (45790us total) trans 4294967821 rc 0/0 00000100:00100000:2.0:1789762862.913824:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 504 00000100:00100000:2.0:1789762862.913834:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336148916648 time 336149025912 state 0x2 ref 0 00000100:00100000:0.0:1789762862.916291:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762862.916387:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067431680 00000100:00100000:1.0:1789762862.916397:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067431680 00000100:00100000:1.0:1789762862.916445:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336148916648 time 336149025912 state 0x0 ref 1 00000100:00100000:1.0:1789762862.916458:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336199025912 time 336149025912 state 0x1 ref 1 00000100:00100000:0.0:1789762862.952048:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336199025912 time 336199114131 state 0x0 ref 1 00000100:00100000:0.0:1789762862.952070:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067431680:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762862.952124:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 12582912, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x6007 00010000:00010000:0.0:1789762862.952135:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f1762cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.952155:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:0.0:1789762862.952161:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762862.952169:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f1762cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762862.952187:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f1762cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762862.963062:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967822 is committed 00002000:00010000:1.0:1789762862.963068:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 12582912 -> 13631488 00002000:00010000:1.0:1789762862.963076:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 24583 -> 26631 00010000:00010000:1.0:1789762862.963089:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f1762cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.963099:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f1762cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.963107:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f1762cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.963112:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762862.963116:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f1762cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.963128:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f1762cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.963146:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f1762cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762862.963151:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f1762cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762862.963216:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067431680:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11152us (46925us total) trans 4294967822 rc 0/0 00000100:00100000:1.0:1789762862.963225:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 505 00000100:00100000:1.0:1789762862.963235:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336199025912 time 336199114131 state 0x2 ref 0 00000100:00100000:0.0:1789762862.965278:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762862.965325:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067431808 00000100:00100000:3.0:1789762862.965334:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067431808 00000100:00100000:3.0:1789762862.965392:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336199025912 time 336199114131 state 0x0 ref 1 00000100:00100000:3.0:1789762862.965404:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336249114131 time 336199114131 state 0x1 ref 1 00000100:00100000:1.0:1789762863.002129:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336249114131 time 336249193693 state 0x0 ref 1 00000100:00100000:1.0:1789762863.002167:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067431808:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:3.0:1789762863.002191:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789762863.002222:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 13631488, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x6807 00010000:00010000:1.0:1789762863.002230:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1762d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.002246:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:1.0:1789762863.002251:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762863.002256:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1762d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.002269:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1762d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762863.013506:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 13631488 -> 14680064 00000001:00080000:0.0:1789762863.013510:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967823 is committed 00002000:00010000:1.0:1789762863.013511:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:1.0:1789762863.013525:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1762d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.013535:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1762d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.013542:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1762d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.013547:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762863.013551:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1762d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.013559:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1762d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.013575:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1762d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.013581:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1762d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762863.013638:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067431808:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11493us (48360us total) trans 4294967823 rc 0/0 00000100:00100000:1.0:1789762863.013647:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 506 00000100:00100000:1.0:1789762863.013656:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336249114131 time 336249193693 state 0x2 ref 0 00000100:00100000:3.0:1789762863.015614:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762863.015667:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067431936 00000100:00100000:2.0:1789762863.015677:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067431936 00000100:00100000:2.0:1789762863.015728:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336249114131 time 336249193693 state 0x0 ref 1 00000100:00100000:2.0:1789762863.015745:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336299193693 time 336249193693 state 0x1 ref 1 00000100:00100000:2.0:1789762863.052185:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336299193693 time 336299247899 state 0x0 ref 1 00000100:00100000:2.0:1789762863.052214:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067431936:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762863.052276:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 14680064, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x7007 00010000:00010000:2.0:1789762863.052285:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.052328:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:2.0:1789762863.052334:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762863.052340:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.052354:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762863.063702:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967824 is committed 00002000:00010000:2.0:1789762863.063721:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 14680064 -> 15728640 00002000:00010000:2.0:1789762863.063728:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:2.0:1789762863.063741:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.063752:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.063760:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.063766:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762863.063770:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.063777:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.063804:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.063810:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762863.063876:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067431936:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11673us (48260us total) trans 4294967824 rc 0/0 00000100:00100000:2.0:1789762863.063888:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 507 00000100:00100000:2.0:1789762863.063897:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336299193693 time 336299247899 state 0x2 ref 0 00000100:00100000:2.0:1789762863.065792:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762863.065834:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067432064 00000100:00100000:1.0:1789762863.065844:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067432064 00000100:00100000:1.0:1789762863.065888:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336299193693 time 336299247899 state 0x0 ref 1 00000100:00100000:1.0:1789762863.065901:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336349247899 time 336299247899 state 0x1 ref 1 00000100:00100000:2.0:1789762863.102284:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336349247899 time 336349343412 state 0x0 ref 1 00000100:00100000:2.0:1789762863.102315:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067432064:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762863.102399:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 15728640, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x7807 00010000:00010000:2.0:1789762863.102410:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.102441:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:2.0:1789762863.102446:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762863.102452:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.102490:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762863.116809:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967420 is committed 00010000:00010000:0.0:1789762863.116838:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f175f6f lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1789762863.116846:0:7221:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9c37d176f540 refcount 2 committed 0 result 0 batchid 9 00000020:00080000:1.0:1789762863.116853:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37d1097d00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9c37d176fa80 00000020:00080000:1.0:1789762863.116857:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x2:0x0].9 00010000:00010000:0.0:1789762863.116859:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f175f6f lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1789762863.116860:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37d1097f00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9c37ff865c00 00000020:00080000:1.0:1789762863.116867:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x2:0x0].9 00010000:00010000:0.0:1789762863.116875:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f175f6f lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.116882:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762863.116888:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f175f6f lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762863.116889:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967422 is committed 00000020:00080000:2.0:1789762863.116893:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 9 committed batchid 8 00000020:00080000:1.0:1789762863.116896:0:7221:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9c37eebc60c0 refcount 2 committed 0 result 0 batchid 10 00010000:00010000:0.0:1789762863.116898:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f175f6f lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1789762863.116901:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37e1b83800 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9c37eebc6600 00000020:00080000:1.0:1789762863.116910:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x2:0x0].10 00000020:00080000:1.0:1789762863.116913:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37e1b83300 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9c37ee8bcc00 00000020:00080000:1.0:1789762863.116939:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x2:0x0].10 00010000:00010000:0.0:1789762863.116945:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762863.116949:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762863.116954:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f175f6f lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.116960:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f175f6f lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.116983:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7000/0xdf5a7ac59f175fdf lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789762863.116984:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 15 committed 00000004:00080000:1.0:1789762863.116999:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 15 committed 00010000:00010000:0.0:1789762863.117001:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7000/0xdf5a7ac59f175fdf lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117009:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7000/0xdf5a7ac59f175fdf lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117013:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762863.117016:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7000/0xdf5a7ac59f175fdf lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117021:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7000/0xdf5a7ac59f175fdf lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117030:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762863.117033:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789762863.117036:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7000/0xdf5a7ac59f175fdf lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117040:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc7000/0xdf5a7ac59f175fdf lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117053:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175ff4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117058:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175ff4 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117064:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175ff4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117067:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762863.117074:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175ff4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117080:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175ff4 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117089:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x0 (ffff9c37e125a000) 00010000:00010000:0.0:1789762863.117091:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762863.117093:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175ff4 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117097:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f175ff4 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117116:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176033 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117121:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176033 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117125:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176033 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117129:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768200 already unused, calling callback (ffffffffc1f644f0) 00000020:00080000:2.0:1789762863.117131:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 10 committed batchid 8 00010000:00010000:0.0:1789762863.117132:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176033 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117141:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176033 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117149:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x73550ecb (ffff9c37e125ae00) 00010000:00010000:0.0:1789762863.117151:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000020:00080000:2.0:1789762863.117156:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 8 00010000:00010000:0.0:1789762863.117158:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176033 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117164:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176033 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117178:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f17606b lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762863.117184:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d2a13480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074697984:0@lo:6:osp-syn-0-1.0 00010000:00010000:0.0:1789762863.117188:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f17606b lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117198:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f17606b lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117205:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762863.117210:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f17606b lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117219:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f17606b lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117230:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x2b0e463d (ffff9c37d3bee100) 00010000:00010000:0.0:1789762863.117232:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762863.117235:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f17606b lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762863.117238:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9255500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074698112:0@lo:6:osp-syn-1-1.0 00010000:00010000:0.0:1789762863.117246:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c9bc6a00/0xdf5a7ac59f17606b lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117258:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d000/0xdf5a7ac59f176136 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762863.117259:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1789762863.117276:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d000/0xdf5a7ac59f176136 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117281:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d000/0xdf5a7ac59f176136 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117285:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762863.117288:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d000/0xdf5a7ac59f176136 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117293:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d000/0xdf5a7ac59f176136 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762863.117294:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074697984 00000100:00100000:2.0:1789762863.117303:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074697984 00010000:00010000:0.0:1789762863.117307:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x73550ecb (ffff9c37e125ae00) 00000100:00100000:1.0:1789762863.117313:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1789762863.117314:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762863.117320:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d000/0xdf5a7ac59f176136 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117328:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d273d000/0xdf5a7ac59f176136 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117358:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f176152 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.117364:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f176152 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762863.117367:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074698112 00010000:00010000:0.0:1789762863.117381:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f176152 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762863.117385:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074698112 00000100:00100000:2.0:1789762863.117388:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00010000:00010000:0.0:1789762863.117395:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762863.117405:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f176152 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762863.117406:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074697984:12345-0@lo:6:osp-syn-0-1.0 00010000:00010000:0.0:1789762863.117419:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f176152 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762863.117428:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00010000:00010000:0.0:1789762863.117429:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x11:0x0].0x2b0e463d (ffff9c37d3bee100) 00010000:00010000:0.0:1789762863.117433:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762863.117436:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f176152 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1789762863.117440:0:8054:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:10 count 1 00000100:00100000:1.0:1789762863.117442:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ffccb050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074698112:12345-0@lo:6:osp-syn-1-1.0 00010000:00010000:0.0:1789762863.117448:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f176152 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x11:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00002000:00080000:1.0:1789762863.117460:0:8052:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:10 count 1 00010000:00010000:2.0:1789762863.117466:0:8054:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762863.117485:0:8052:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0xa:0x0] initial LVB size: 8388608, mtime: 0x6aad9d2d, atime: 0x0, ctime: 0x6aad9d2d, blocks: 0x803 00010000:00010000:2.0:1789762863.117488:0:8054:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f1760f7 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c89b expref: 9 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.117494:0:8052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000809719ad/0xdf5a7ac59f1762ef lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.117498:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xa:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762863.117507:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xa:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762863.117509:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xa:0x0].0x0 since=336s v=(0 0) 00010000:00010000:2.0:1789762863.117512:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.117513:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xa:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762863.117524:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000809719ad/0xdf5a7ac59f1762ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.117537:0:8052:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000809719ad/0xdf5a7ac59f1762ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.117543:0:8054:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f1760f7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c89b expref: 9 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.117546:0:8052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000809719ad/0xdf5a7ac59f1762ef lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.117560:0:8052:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000809719ad/0xdf5a7ac59f1762ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.117566:0:8054:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f1760f7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c89b expref: 9 pid: 8053 timeout: 436 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.117585:0:8052:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1789762863.117585:0:8054:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d9256680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8054:1876702074698240:192.168.203.37@tcp:104: 00010000:00010000:3.0:1789762863.117600:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762863.117610:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000809719ad/0xdf5a7ac59f1762ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.117622:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762863.117628:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000809719ad/0xdf5a7ac59f1762ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.117648:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000809719ad/0xdf5a7ac59f1762ef lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762863.117655:0:8054:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c9739780 going to sleep for 11 seconds 00010000:00010000:3.0:1789762863.117667:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000809719ad/0xdf5a7ac59f1762ef lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789762863.117671:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1789762863.117678:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000809719ad/0xdf5a7ac59f1762ef lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789762863.117680:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762863.117702:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762863.117705:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762863.117708:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762863.118115:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967825 is committed 00002000:00010000:0.0:1789762863.118117:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 15728640 -> 16777216 00002000:00010000:0.0:1789762863.118122:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 30727 -> 32775 00010000:00010000:0.0:1789762863.118130:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.118138:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.118145:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.118150:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762863.118152:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.118167:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.118183:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.118192:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1762e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762863.118209:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ffccb050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074698112:12345-0@lo:6:osp-syn-1-1.0 Request processed in 770us (897us total) trans 4294967398 rc 0/0 00000100:00100000:1.0:1789762863.118217:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000004:00080000:2.0:1789762863.118236:0:3424:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37d9255500/2, rc 0, transno 4294967398 00000100:00100000:2.0:1789762863.118254:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9255500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074698112:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1789762863.118261:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067432064:12345-192.168.203.37@tcp:4:dd.500 Request processed in 15957us (52468us total) trans 4294967825 rc 0/0 00000100:00100000:0.0:1789762863.118272:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 508 00000100:00100000:0.0:1789762863.118283:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336349247899 time 336349343412 state 0x2 ref 0 00000100:00100000:2.0:1789762863.118498:0:8054:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d9256680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8054:1876702074698240:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762863.118517:0:8054:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762863.118715:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762863.118743:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067432192 00000100:00100000:2.0:1789762863.118747:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067432192 00010000:00010000:2.0:1789762863.118765:0:7218:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f1760f7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c89b expref: 6 pid: 8053 timeout: 436 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762863.118787:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 80 00000100:00100000:2.0:1789762863.118792:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067432192:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762863.118801:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762863.118805:0:7218:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f1760f7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c89b expref: 6 pid: 8053 timeout: 436 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.118814:0:7218:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f1760f7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c89b expref: 6 pid: 8053 timeout: 436 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.118837:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f1760f7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c89b expref: 6 pid: 8053 timeout: 436 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.118845:0:7218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xa:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762863.118848:0:7218:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xa:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762863.118851:0:7218:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.118857:0:7218:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.118870:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762863.118888:0:8054:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762863.118896:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067432192:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 102us (180us total) trans 0 rc 0/0 00010000:00010000:0.0:1789762863.118910:0:8054:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762863.118910:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 80 00010000:00010000:0.0:1789762863.118945:0:8054:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.118955:0:8054:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.118978:0:8054:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762863.118986:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762863.118993:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.119000:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762863.119004:0:6408:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.119011:0:6408:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.119032:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 1/0,0 mode: --/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.119038:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f1762e8 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.119056:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762863.119059:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762863.119061:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762863.119448:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074697984:12345-0@lo:6:osp-syn-0-1.0 Request processed in 2048us (2189us total) trans 4294967826 rc 0/0 00000100:00100000:0.0:1789762863.119455:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000004:00080000:3.0:1789762863.119473:0:3427:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c36d2a13480/2, rc 0, transno 4294967826 00000100:00100000:3.0:1789762863.119488:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d2a13480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074697984:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1789762863.120236:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762863.120281:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067432320 00000100:00100000:1.0:1789762863.120287:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067432320 00000100:00100000:1.0:1789762863.120317:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336349247899 time 336349343412 state 0x0 ref 1 00000100:00100000:1.0:1789762863.120328:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336399343412 time 336349343412 state 0x1 ref 1 00000100:00100000:1.0:1789762863.152324:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336399343412 time 336399391969 state 0x0 ref 1 00000100:00100000:1.0:1789762863.152345:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067432320:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762863.152412:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 16777216, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x8007 00010000:00010000:1.0:1789762863.152419:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.152431:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:1.0:1789762863.152434:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762863.152439:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.152449:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762863.168612:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967826 is committed 00000001:00080000:2.0:1789762863.168629:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967827 is committed 00002000:00010000:1.0:1789762863.168641:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 16777216 -> 17825792 00002000:00010000:1.0:1789762863.168648:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:1.0:1789762863.168665:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.168678:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.168685:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.168691:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762863.168696:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.168716:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.168734:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.168739:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762863.168852:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067432320:12345-192.168.203.37@tcp:4:dd.500 Request processed in 16508us (48611us total) trans 4294967827 rc 0/0 00000100:00100000:1.0:1789762863.168863:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 509 00000100:00100000:1.0:1789762863.168876:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336399343412 time 336399391969 state 0x2 ref 0 00000100:00100000:0.0:1789762863.171893:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762863.171971:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067432448 00000100:00100000:2.0:1789762863.171982:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067432448 00000100:00100000:2.0:1789762863.172034:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336399343412 time 336399391969 state 0x0 ref 1 00000100:00100000:2.0:1789762863.172047:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336449391969 time 336399391969 state 0x1 ref 1 00000100:00100000:1.0:1789762863.202456:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336449391969 time 336449511722 state 0x0 ref 1 00000100:00100000:1.0:1789762863.202492:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067432448:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762863.202563:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 17825792, mtime: 0x6aad9d2e, atime: 0x0, ctime: 0x6aad9d2e, blocks: 0x8807 00010000:00010000:1.0:1789762863.202573:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.202591:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:1.0:1789762863.202595:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762863.202601:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.202615:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762863.216316:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967828 is committed 00002000:00010000:1.0:1789762863.216331:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 17825792 -> 18874368 00002000:00010000:1.0:1789762863.216337:0:13300:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb mtime from disk: 1789762862 -> 1789762863 00002000:00010000:1.0:1789762863.216339:0:13300:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb ctime from disk: 1789762862 -> 1789762863 00002000:00010000:1.0:1789762863.216342:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 34823 -> 36871 00010000:00010000:1.0:1789762863.216356:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.216387:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.216395:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.216400:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762863.216416:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.216423:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.216445:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.216451:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1762fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762863.216552:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067432448:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14072us (44654us total) trans 4294967828 rc 0/0 00000100:00100000:1.0:1789762863.216562:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 510 00000100:00100000:1.0:1789762863.216574:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336449391969 time 336449511722 state 0x2 ref 0 00000100:00100000:0.0:1789762863.220126:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762863.220200:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067432576 00000100:00100000:2.0:1789762863.220211:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067432576 00000100:00100000:2.0:1789762863.220281:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336449391969 time 336449511722 state 0x0 ref 1 00000100:00100000:2.0:1789762863.220296:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336499511722 time 336449511722 state 0x1 ref 1 00000100:00100000:2.0:1789762863.252547:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336499511722 time 336499605316 state 0x0 ref 1 00000100:00100000:2.0:1789762863.252606:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067432576:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762863.252682:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 18874368, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0x9007 00010000:00010000:2.0:1789762863.252692:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f176304 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.252712:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:2.0:1789762863.252717:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762863.252723:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f176304 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.252738:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f176304 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762863.270480:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967829 is committed 00002000:00010000:1.0:1789762863.270584:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:1.0:1789762863.270590:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:1.0:1789762863.270603:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f176304 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.270614:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f176304 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.270621:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f176304 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.270627:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762863.270631:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f176304 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.270639:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f176304 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.270658:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f176304 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.270663:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c40f06e/0xdf5a7ac59f176304 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762863.270752:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067432576:12345-192.168.203.37@tcp:4:dd.500 Request processed in 18155us (50632us total) trans 4294967829 rc 0/0 00000100:00100000:1.0:1789762863.270762:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 511 00000100:00100000:1.0:1789762863.270774:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336499511722 time 336499605316 state 0x2 ref 0 00000100:00100000:0.0:1789762863.274804:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762863.274893:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067432704 00000100:00100000:1.0:1789762863.274906:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067432704 00000100:00100000:1.0:1789762863.274982:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336499511722 time 336499605316 state 0x0 ref 1 00000100:00100000:1.0:1789762863.274998:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336549605316 time 336499605316 state 0x1 ref 1 00000100:00100000:3.0:1789762863.302601:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336549605316 time 336549660430 state 0x0 ref 1 00000100:00100000:3.0:1789762863.302633:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff687480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067432704:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762863.302722:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 19922944, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0x9807 00010000:00010000:3.0:1789762863.302732:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f17630b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.302751:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:3.0:1789762863.302756:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762863.302779:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f17630b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.302795:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f17630b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762863.320123:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 19922944 -> 20971520 00002000:00010000:1.0:1789762863.320132:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 38919 -> 40967 00000001:00080000:3.0:1789762863.320133:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967830 is committed 00010000:00010000:1.0:1789762863.320148:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f17630b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.320163:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f17630b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.320175:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f17630b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.320186:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762863.320193:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f17630b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.320203:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f17630b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.320227:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f17630b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.320238:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f17630b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762863.320331:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff687480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067432704:12345-192.168.203.37@tcp:4:dd.500 Request processed in 17706us (45535us total) trans 4294967830 rc 0/0 00000100:00100000:1.0:1789762863.320340:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 512 00000100:00100000:1.0:1789762863.320353:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336549605316 time 336549660430 state 0x2 ref 0 00000100:00100000:0.0:1789762863.324683:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762863.324743:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067432832 00000100:00100000:1.0:1789762863.324756:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067432832 00000100:00100000:1.0:1789762863.324812:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336549605316 time 336549660430 state 0x0 ref 1 00000100:00100000:1.0:1789762863.324825:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336599660430 time 336549660430 state 0x1 ref 1 00000100:00100000:1.0:1789762863.352688:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336599660430 time 336599723449 state 0x0 ref 1 00000100:00100000:1.0:1789762863.352741:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067432832:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762863.352834:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 20971520, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0xa007 00010000:00010000:1.0:1789762863.352847:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f176312 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.352871:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:1.0:1789762863.352877:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762863.352904:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f176312 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.352952:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f176312 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762863.374082:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:1.0:1789762863.374091:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:1.0:1789762863.374105:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f176312 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762863.374116:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967831 is committed 00010000:00010000:1.0:1789762863.374118:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f176312 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.374129:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f176312 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.374136:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762863.374141:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f176312 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.374149:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f176312 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.374171:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f176312 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.374182:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f176312 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762863.374284:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067432832:12345-192.168.203.37@tcp:4:dd.500 Request processed in 21568us (49605us total) trans 4294967831 rc 0/0 00000100:00100000:1.0:1789762863.374298:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 513 00000100:00100000:1.0:1789762863.374311:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336599660430 time 336599723449 state 0x2 ref 0 00000100:00100000:0.0:1789762863.378206:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762863.378275:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067432960 00000100:00100000:1.0:1789762863.378291:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067432960 00000100:00100000:1.0:1789762863.378351:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336599660430 time 336599723449 state 0x0 ref 1 00000100:00100000:1.0:1789762863.378366:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336649723449 time 336599723449 state 0x1 ref 1 00000100:00100000:2.0:1789762863.402760:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336649723449 time 336649822469 state 0x0 ref 1 00000100:00100000:2.0:1789762863.402789:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067432960:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762863.402857:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 22020096, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0xa807 00010000:00010000:2.0:1789762863.402867:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176319 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.402894:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:2.0:1789762863.402899:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762863.402906:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.402962:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762863.424997:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 22020096 -> 23068672 00002000:00010000:3.0:1789762863.425007:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:3.0:1789762863.425023:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.425036:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176319 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.425044:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176319 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.425049:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762863.425054:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176319 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.425063:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176319 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.425086:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176319 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.425102:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176319 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762863.425223:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067432960:12345-192.168.203.37@tcp:4:dd.500 Request processed in 22440us (47023us total) trans 4294967832 rc 0/0 00000100:00100000:3.0:1789762863.425235:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 514 00000100:00100000:3.0:1789762863.425248:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336649723449 time 336649822469 state 0x2 ref 0 00000001:00080000:3.0:1789762863.425300:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967832 is committed 00000100:00100000:0.0:1789762863.429357:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762863.429486:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067433088 00000100:00100000:1.0:1789762863.429502:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067433088 00000100:00100000:1.0:1789762863.429580:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336649723449 time 336649822469 state 0x0 ref 1 00000100:00100000:1.0:1789762863.429600:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336699822469 time 336649822469 state 0x1 ref 1 00000100:00100000:2.0:1789762863.452880:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336699822469 time 336699934069 state 0x0 ref 1 00000100:00100000:2.0:1789762863.452948:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067433088:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762863.453030:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 23068672, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0xb007 00010000:00010000:2.0:1789762863.453041:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176320 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.453061:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:2.0:1789762863.453067:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762863.453082:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.453100:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762863.466262:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967833 is committed 00002000:00010000:0.0:1789762863.466269:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:0.0:1789762863.466277:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:0.0:1789762863.466292:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.466304:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176320 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.466310:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176320 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.466316:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762863.466320:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176320 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.466337:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176320 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.466353:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176320 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.466358:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176320 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762863.466449:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067433088:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13544us (37097us total) trans 4294967833 rc 0/0 00000100:00100000:0.0:1789762863.466458:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 515 00000100:00100000:0.0:1789762863.466468:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336699822469 time 336699934069 state 0x2 ref 0 00000100:00100000:0.0:1789762863.468618:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762863.468660:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067433216 00000100:00100000:1.0:1789762863.468669:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067433216 00000100:00100000:1.0:1789762863.468709:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336699822469 time 336699934069 state 0x0 ref 1 00000100:00100000:1.0:1789762863.468720:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336749934069 time 336699934069 state 0x1 ref 1 00000100:00100000:3.0:1789762863.503003:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336749934069 time 336750060914 state 0x0 ref 1 00000100:00100000:3.0:1789762863.503030:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067433216:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762863.503090:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 24117248, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0xb807 00010000:00010000:3.0:1789762863.503098:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f176327 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.503114:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:3.0:1789762863.503119:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762863.503125:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f176327 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.503138:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f176327 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762863.515602:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967834 is committed 00002000:00010000:0.0:1789762863.515622:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 24117248 -> 25165824 00002000:00010000:0.0:1789762863.515628:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:0.0:1789762863.515638:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f176327 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.515647:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f176327 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.515653:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f176327 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.515668:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762863.515672:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f176327 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.515678:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f176327 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.515690:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f176327 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.515696:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f176327 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762863.515753:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067433216:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12730us (47134us total) trans 4294967834 rc 0/0 00000100:00100000:0.0:1789762863.515762:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 516 00000100:00100000:0.0:1789762863.515770:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336749934069 time 336750060914 state 0x2 ref 0 00000100:00100000:0.0:1789762863.517817:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762863.517862:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067433344 00000100:00100000:3.0:1789762863.517873:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067433344 00000100:00100000:3.0:1789762863.517944:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336749934069 time 336750060914 state 0x0 ref 1 00000100:00100000:3.0:1789762863.517958:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336800060914 time 336750060914 state 0x1 ref 1 00000100:00100000:0.0:1789762863.553092:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336800060914 time 336800162258 state 0x0 ref 1 00000100:00100000:0.0:1789762863.553116:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067433344:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762863.553159:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 25165824, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0xc007 00010000:00010000:0.0:1789762863.553179:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f17632e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.553192:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:0.0:1789762863.553196:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762863.553201:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f17632e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.553211:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f17632e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762863.559604:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762863.559664:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067433472 00000100:00100000:0.0:1789762863.559672:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762863.559676:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067433472 00000100:00100000:0.0:1789762863.559709:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762863.559766:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 357 00000100:00100000:0.0:1789762863.559774:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762863.559790:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d157a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067433472:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762863.559826:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762863.559882:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d157a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067433472:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 95us (275us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762863.559901:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762863.559970:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 357 00000100:00100000:3.0:1789762863.559998:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067433728 00000100:00100000:3.0:1789762863.560005:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067433728 00000100:00100000:0.0:1789762863.560064:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067433984 00000100:00100000:0.0:1789762863.560071:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067433984 00000100:00100000:0.0:1789762863.560131:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067433600 00000100:00100000:0.0:1789762863.560135:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067433600 00000100:00100000:0.0:1789762863.560183:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067434112 00000100:00100000:0.0:1789762863.560187:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067434112 00000100:00100000:2.0:1789762863.560249:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 478 00000100:00100000:0.0:1789762863.560263:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 479 00000100:00100000:2.0:1789762863.560269:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067433728:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762863.560282:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067433984:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762863.560291:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 480 00000100:00100000:3.0:1789762863.560305:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:2398:x1876702067433600:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762863.560320:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067433728:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 60us (648us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762863.560331:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 478 00000100:00100000:0.0:1789762863.560350:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067433984:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 72us (641us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762863.560363:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:2398:x1876702067433600:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 59us (606us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762863.560380:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 479 00000100:00100000:2.0:1789762863.560400:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 481 00000100:00100000:3.0:1789762863.560411:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 480 00000100:00100000:2.0:1789762863.560415:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067434112:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762863.560430:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067433856 00000100:00100000:0.0:1789762863.560437:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067433856 00000100:00100000:2.0:1789762863.560475:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067434112:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 59us (578us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762863.560492:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 481 00000100:00100000:0.0:1789762863.560519:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 193 00000100:00100000:0.0:1789762863.560535:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2398:x1876702067433856:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:2.0:1789762863.560548:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762863.560557:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762863.560564:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762863.560590:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2398:x1876702067433856:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 60us (762us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762863.560601:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 193 00000001:00080000:3.0:1789762863.566128:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967835 is committed 00002000:00010000:0.0:1789762863.566143:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:0.0:1789762863.566151:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:0.0:1789762863.566167:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f17632e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.566180:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f17632e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.566188:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f17632e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.566194:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762863.566197:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f17632e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.566203:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f17632e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.566216:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f17632e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.566232:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f17632e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762863.566277:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067433344:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13167us (48461us total) trans 4294967835 rc 0/0 00000100:00100000:0.0:1789762863.566284:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 517 00000100:00100000:0.0:1789762863.566292:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336800060914 time 336800162258 state 0x2 ref 0 00000100:00100000:2.0:1789762863.568383:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762863.568425:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067434240 00000100:00100000:0.0:1789762863.568430:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067434240 00000100:00100000:0.0:1789762863.568457:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336800060914 time 336800162258 state 0x0 ref 1 00000100:00100000:0.0:1789762863.568466:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336850162258 time 336800162258 state 0x1 ref 1 00000100:00100000:2.0:1789762863.603181:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336850162258 time 336850245872 state 0x0 ref 1 00000100:00100000:2.0:1789762863.603206:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9256680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067434240:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762863.603261:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 26214400, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0xc807 00010000:00010000:2.0:1789762863.603270:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176335 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.603286:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:2.0:1789762863.603291:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762863.603307:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176335 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762863.603320:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176335 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762863.616058:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 26214400 -> 27262976 00002000:00010000:3.0:1789762863.616065:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:3.0:1789762863.616078:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176335 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.616089:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176335 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.616096:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176335 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.616113:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762863.616117:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176335 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.616124:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176335 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.616140:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176335 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.616146:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176335 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762863.616220:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9256680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067434240:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13022us (47849us total) trans 4294967836 rc 0/0 00000100:00100000:3.0:1789762863.616232:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 518 00000100:00100000:3.0:1789762863.616242:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336850162258 time 336850245872 state 0x2 ref 0 00000001:00080000:3.0:1789762863.616280:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967836 is committed 00000100:00100000:1.0:1789762863.618431:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762863.618481:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067434368 00000100:00100000:0.0:1789762863.618490:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067434368 00000100:00100000:0.0:1789762863.618532:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336850162258 time 336850245872 state 0x0 ref 1 00000100:00100000:0.0:1789762863.618543:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336900245872 time 336850245872 state 0x1 ref 1 00000100:00100000:1.0:1789762863.653283:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336900245872 time 336900353201 state 0x0 ref 1 00000100:00100000:1.0:1789762863.653303:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067434368:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762863.653347:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 27262976, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0xd007 00010000:00010000:1.0:1789762863.653354:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17633c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.653381:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:1.0:1789762863.653387:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762863.653392:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17633c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.653404:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17633c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762863.666834:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967837 is committed 00002000:00010000:0.0:1789762863.666912:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:0.0:1789762863.666937:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 53255 -> 55303 00010000:00010000:0.0:1789762863.666950:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17633c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.666977:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17633c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.666984:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17633c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.666990:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762863.666993:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17633c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.666999:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17633c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.667013:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17633c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.667019:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17633c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762863.667080:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067434368:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13785us (48653us total) trans 4294967837 rc 0/0 00000100:00100000:0.0:1789762863.667089:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 519 00000100:00100000:0.0:1789762863.667098:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336900245872 time 336900353201 state 0x2 ref 0 00000100:00100000:2.0:1789762863.669340:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762863.669395:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067434496 00000100:00100000:0.0:1789762863.669411:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067434496 00000100:00100000:0.0:1789762863.669450:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336900245872 time 336900353201 state 0x0 ref 1 00000100:00100000:0.0:1789762863.669459:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336950353201 time 336900353201 state 0x1 ref 1 00000100:00100000:1.0:1789762863.703440:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336950353201 time 336950505897 state 0x0 ref 1 00000100:00100000:1.0:1789762863.703466:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9257480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067434496:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762863.703515:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 28311552, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0xd807 00010000:00010000:1.0:1789762863.703523:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176343 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.703536:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=336s v=(0 0) 00010000:00010000:1.0:1789762863.703540:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762863.703545:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.703557:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762863.716561:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967838 is committed 00002000:00010000:0.0:1789762863.716635:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 28311552 -> 29360128 00002000:00010000:0.0:1789762863.716640:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 55303 -> 57351 00010000:00010000:0.0:1789762863.716651:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.716661:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176343 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.716668:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176343 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.716673:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762863.716676:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176343 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.716683:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176343 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.716698:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176343 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.716703:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176343 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762863.716778:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9257480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067434496:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13321us (47440us total) trans 4294967838 rc 0/0 00000100:00100000:0.0:1789762863.716787:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 520 00000100:00100000:0.0:1789762863.716797:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336950353201 time 336950505897 state 0x2 ref 0 00000100:00100000:2.0:1789762863.719054:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762863.719106:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067434624 00000100:00100000:3.0:1789762863.719117:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067434624 00000100:00100000:3.0:1789762863.719173:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 336950353201 time 336950505897 state 0x0 ref 1 00000100:00100000:3.0:1789762863.719198:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337000505897 time 336950505897 state 0x1 ref 1 00010000:00010000:1.0:1789762863.744034:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762863.753492:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337000505897 time 337000559798 state 0x0 ref 1 00000100:00100000:3.0:1789762863.753516:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9256a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067434624:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762863.753571:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 29360128, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0xe007 00010000:00010000:3.0:1789762863.753580:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17634a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.753596:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:3.0:1789762863.753601:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762863.753607:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17634a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.753620:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17634a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762863.766041:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967839 is committed 00002000:00010000:0.0:1789762863.766145:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 29360128 -> 30408704 00002000:00010000:0.0:1789762863.766152:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 57351 -> 59399 00010000:00010000:0.0:1789762863.766166:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17634a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.766180:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17634a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.766187:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17634a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.766192:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762863.766197:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17634a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.766206:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17634a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.766227:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17634a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.766249:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17634a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762863.766317:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9256a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067434624:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12807us (47265us total) trans 4294967839 rc 0/0 00000100:00100000:0.0:1789762863.766330:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 521 00000100:00100000:0.0:1789762863.766342:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337000505897 time 337000559798 state 0x2 ref 0 00000100:00100000:1.0:1789762863.768499:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762863.768535:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067434752 00000100:00100000:0.0:1789762863.768543:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067434752 00000100:00100000:0.0:1789762863.768582:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337000505897 time 337000559798 state 0x0 ref 1 00000100:00100000:0.0:1789762863.768593:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337050559798 time 337000559798 state 0x1 ref 1 00000100:00100000:3.0:1789762863.803621:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337050559798 time 337050682366 state 0x0 ref 1 00000100:00100000:3.0:1789762863.803654:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067434752:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762863.803721:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 30408704, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0xe807 00010000:00010000:3.0:1789762863.803730:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f176351 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.803773:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:3.0:1789762863.803777:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762863.803783:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f176351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.803797:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f176351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762863.816967:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967840 is committed 00002000:00010000:0.0:1789762863.816987:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 30408704 -> 31457280 00002000:00010000:0.0:1789762863.816996:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 59399 -> 61447 00010000:00010000:0.0:1789762863.817011:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f176351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.817025:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f176351 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.817035:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f176351 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.817058:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273dc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762863.817064:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f176351 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.817074:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f176351 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.817095:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f176351 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.817103:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000322d2d0d/0xdf5a7ac59f176351 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762863.817200:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067434752:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13556us (48696us total) trans 4294967840 rc 0/0 00000100:00100000:0.0:1789762863.817216:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 522 00000100:00100000:0.0:1789762863.817230:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337050559798 time 337050682366 state 0x2 ref 0 00000100:00100000:1.0:1789762863.819780:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762863.819847:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067434880 00000100:00100000:0.0:1789762863.819856:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067434880 00000100:00100000:0.0:1789762863.819894:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337050559798 time 337050682366 state 0x0 ref 1 00000100:00100000:0.0:1789762863.819905:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337100682366 time 337050682366 state 0x1 ref 1 00000100:00100000:0.0:1789762863.853694:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337100682366 time 337100758940 state 0x0 ref 1 00000100:00100000:0.0:1789762863.853734:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067434880:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762863.853784:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 31457280, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0xf007 00010000:00010000:0.0:1789762863.853791:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176358 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.853805:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:0.0:1789762863.853809:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762863.853813:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176358 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.853824:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176358 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762863.867730:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967841 is committed 00002000:00010000:0.0:1789762863.867749:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 31457280 -> 32505856 00002000:00010000:0.0:1789762863.867756:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 61447 -> 63495 00010000:00010000:0.0:1789762863.867767:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176358 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.867781:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176358 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.867792:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176358 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.867800:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762863.867813:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176358 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.867823:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176358 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.867837:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176358 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.867843:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f176358 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762863.867904:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067434880:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14177us (48123us total) trans 4294967841 rc 0/0 00000100:00100000:0.0:1789762863.867912:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 523 00000100:00100000:0.0:1789762863.867965:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337100682366 time 337100758940 state 0x2 ref 0 00000100:00100000:2.0:1789762863.870133:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762863.870178:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067435008 00000100:00100000:0.0:1789762863.870189:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067435008 00000100:00100000:0.0:1789762863.870233:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337100682366 time 337100758940 state 0x0 ref 1 00000100:00100000:0.0:1789762863.870248:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337150758940 time 337100758940 state 0x1 ref 1 00000100:00100000:1.0:1789762863.903774:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337150758940 time 337150835516 state 0x0 ref 1 00000100:00100000:1.0:1789762863.903806:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9257100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067435008:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762863.903857:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 32505856, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0xf807 00010000:00010000:1.0:1789762863.903864:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f17635f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.903878:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:1.0:1789762863.903883:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762863.903890:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f17635f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762863.903904:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f17635f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762863.917616:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967842 is committed 00002000:00010000:0.0:1789762863.917631:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 32505856 -> 33554432 00002000:00010000:0.0:1789762863.917637:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 63495 -> 65543 00010000:00010000:0.0:1789762863.917648:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f17635f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.917665:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f17635f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.917672:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f17635f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.917677:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762863.917681:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f17635f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.917687:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f17635f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.917702:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f17635f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.917707:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f17635f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762863.917777:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9257100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067435008:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13982us (47647us total) trans 4294967842 rc 0/0 00000100:00100000:0.0:1789762863.917787:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 524 00000100:00100000:0.0:1789762863.917797:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337150758940 time 337150835516 state 0x2 ref 0 00000100:00100000:2.0:1789762863.920561:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762863.920603:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067435136 00000100:00100000:0.0:1789762863.920618:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067435136 00000100:00100000:0.0:1789762863.920663:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337150758940 time 337150835516 state 0x0 ref 1 00000100:00100000:0.0:1789762863.920674:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337200835516 time 337150835516 state 0x1 ref 1 00000100:00100000:3.0:1789762863.953896:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337200835516 time 337200952573 state 0x0 ref 1 00000100:00100000:3.0:1789762863.953966:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067435136:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762863.954037:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 33554432, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0x10007 00010000:00010000:3.0:1789762863.954049:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f176366 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.954069:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:3.0:1789762863.954074:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762863.954083:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f176366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762863.954100:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f176366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762863.971139:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967843 is committed 00002000:00010000:0.0:1789762863.971249:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:0.0:1789762863.971256:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 65543 -> 67591 00010000:00010000:0.0:1789762863.971267:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f176366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.971277:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f176366 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.971284:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f176366 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.971289:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762863.971293:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f176366 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.971299:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f176366 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.971331:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f176366 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762863.971339:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000297d49c4/0xdf5a7ac59f176366 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762863.971473:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067435136:12345-192.168.203.37@tcp:4:dd.500 Request processed in 17553us (50914us total) trans 4294967843 rc 0/0 00000100:00100000:0.0:1789762863.971486:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 525 00000100:00100000:0.0:1789762863.971498:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337200835516 time 337200952573 state 0x2 ref 0 00000100:00100000:2.0:1789762863.974387:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762863.974454:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067435264 00000100:00100000:0.0:1789762863.974465:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067435264 00000100:00100000:0.0:1789762863.974534:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337200835516 time 337200952573 state 0x0 ref 1 00000100:00100000:0.0:1789762863.974550:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337250952573 time 337200952573 state 0x1 ref 1 00000100:00100000:3.0:1789762864.004109:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337250952573 time 337251164254 state 0x0 ref 1 00000100:00100000:3.0:1789762864.004159:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9254e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067435264:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762864.004264:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 34603008, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0x10807 00010000:00010000:3.0:1789762864.004275:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17636d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.004302:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:3.0:1789762864.004309:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762864.004328:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17636d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.004346:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17636d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762864.020097:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967844 is committed 00002000:00010000:0.0:1789762864.020103:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:0.0:1789762864.020110:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 67591 -> 69639 00010000:00010000:0.0:1789762864.020125:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17636d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.020137:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17636d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.020145:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17636d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.020151:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762864.020155:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17636d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.020172:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17636d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.020189:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17636d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.020195:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d9497b6/0xdf5a7ac59f17636d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762864.020276:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9254e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067435264:12345-192.168.203.37@tcp:4:dd.500 Request processed in 16130us (45904us total) trans 4294967844 rc 0/0 00000100:00100000:0.0:1789762864.020286:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 526 00000100:00100000:0.0:1789762864.020298:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337250952573 time 337251164254 state 0x2 ref 0 00000100:00100000:1.0:1789762864.023122:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762864.023177:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067435392 00000100:00100000:0.0:1789762864.023191:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067435392 00000100:00100000:0.0:1789762864.023264:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337250952573 time 337251164254 state 0x0 ref 1 00000100:00100000:0.0:1789762864.023279:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337301164254 time 337251164254 state 0x1 ref 1 00000100:00100000:3.0:1789762864.054212:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337301164254 time 337301270452 state 0x0 ref 1 00000100:00100000:3.0:1789762864.054244:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067435392:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762864.054315:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 35651584, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0x11007 00010000:00010000:3.0:1789762864.054326:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f176374 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.054346:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:3.0:1789762864.054352:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762864.054390:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f176374 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.054408:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f176374 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762864.070776:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967845 is committed 00002000:00010000:0.0:1789762864.070841:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 35651584 -> 36700160 00002000:00010000:0.0:1789762864.070847:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 69639 -> 71687 00010000:00010000:0.0:1789762864.070859:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f176374 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.070885:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f176374 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.070893:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f176374 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.070899:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762864.070903:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f176374 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.070910:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f176374 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.070948:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f176374 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.070954:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089e70481/0xdf5a7ac59f176374 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762864.071052:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067435392:12345-192.168.203.37@tcp:4:dd.500 Request processed in 16801us (47914us total) trans 4294967845 rc 0/0 00000100:00100000:0.0:1789762864.071063:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 527 00000100:00100000:0.0:1789762864.071075:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337301164254 time 337301270452 state 0x2 ref 0 00000100:00100000:2.0:1789762864.073816:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762864.073910:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067435520 00000100:00100000:0.0:1789762864.073955:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067435520 00000100:00100000:0.0:1789762864.074018:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337301164254 time 337301270452 state 0x0 ref 1 00000100:00100000:0.0:1789762864.074030:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337351270452 time 337301270452 state 0x1 ref 1 00000100:00100000:3.0:1789762864.104344:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337351270452 time 337351399054 state 0x0 ref 1 00000100:00100000:3.0:1789762864.104398:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9255c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067435520:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762864.104472:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 36700160, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0x11807 00010000:00010000:3.0:1789762864.104483:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17637b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.104502:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:3.0:1789762864.104509:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762864.104517:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17637b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.104535:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17637b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762864.121255:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 36700160 -> 37748736 00002000:00010000:3.0:1789762864.121264:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 71687 -> 73735 00010000:00010000:3.0:1789762864.121281:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17637b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762864.121305:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967846 is committed 00010000:00010000:3.0:1789762864.121307:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17637b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.121328:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17637b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.121335:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762864.121341:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17637b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.121349:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17637b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.121382:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17637b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.121388:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17637b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762864.121505:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9255c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067435520:12345-192.168.203.37@tcp:4:dd.500 Request processed in 17137us (47690us total) trans 4294967846 rc 0/0 00000100:00100000:3.0:1789762864.121515:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 528 00000100:00100000:3.0:1789762864.121528:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337351270452 time 337351399054 state 0x2 ref 0 00000100:00100000:0.0:1789762864.125750:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762864.125796:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067435648 00000100:00100000:3.0:1789762864.125812:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067435648 00000100:00100000:3.0:1789762864.125863:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337351270452 time 337351399054 state 0x0 ref 1 00000100:00100000:3.0:1789762864.125877:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337401399054 time 337351399054 state 0x1 ref 1 00000100:00100000:3.0:1789762864.154415:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337401399054 time 337401468372 state 0x0 ref 1 00000100:00100000:3.0:1789762864.154439:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067435648:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762864.154491:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 37748736, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0x12007 00010000:00010000:3.0:1789762864.154499:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176382 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.154516:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:3.0:1789762864.154520:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762864.154526:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.154539:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762864.174914:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967847 is committed 00002000:00010000:2.0:1789762864.175073:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:2.0:1789762864.175080:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 73735 -> 75783 00010000:00010000:2.0:1789762864.175093:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.175108:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176382 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.175116:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176382 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.175122:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762864.175126:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176382 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.175145:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176382 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.175165:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176382 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.175171:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176382 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762864.175254:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067435648:12345-192.168.203.37@tcp:4:dd.500 Request processed in 20821us (49508us total) trans 4294967847 rc 0/0 00000100:00100000:2.0:1789762864.175263:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 529 00000100:00100000:2.0:1789762864.175272:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337401399054 time 337401468372 state 0x2 ref 0 00000100:00100000:2.0:1789762864.179424:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762864.179491:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067435776 00000100:00100000:0.0:1789762864.179505:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067435776 00000100:00100000:0.0:1789762864.179591:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337401399054 time 337401468372 state 0x0 ref 1 00000100:00100000:0.0:1789762864.179629:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337451468372 time 337401468372 state 0x1 ref 1 00000100:00100000:3.0:1789762864.204520:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337451468372 time 337451585393 state 0x0 ref 1 00000100:00100000:3.0:1789762864.204562:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067435776:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762864.204633:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 38797312, mtime: 0x6aad9d2f, atime: 0x0, ctime: 0x6aad9d2f, blocks: 0x12807 00010000:00010000:3.0:1789762864.204643:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c82dd92b/0xdf5a7ac59f176389 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.204669:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:3.0:1789762864.204674:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762864.204682:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c82dd92b/0xdf5a7ac59f176389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.204697:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c82dd92b/0xdf5a7ac59f176389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762864.223270:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 38797312 -> 39845888 00002000:00010000:2.0:1789762864.223280:0:14171:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb mtime from disk: 1789762863 -> 1789762864 00002000:00010000:2.0:1789762864.223283:0:14171:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb ctime from disk: 1789762863 -> 1789762864 00002000:00010000:2.0:1789762864.223286:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 75783 -> 77831 00010000:00010000:2.0:1789762864.223300:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c82dd92b/0xdf5a7ac59f176389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.223315:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c82dd92b/0xdf5a7ac59f176389 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.223341:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c82dd92b/0xdf5a7ac59f176389 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.223350:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273ca00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762864.223356:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c82dd92b/0xdf5a7ac59f176389 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.223381:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c82dd92b/0xdf5a7ac59f176389 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.223406:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c82dd92b/0xdf5a7ac59f176389 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.223417:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c82dd92b/0xdf5a7ac59f176389 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762864.223516:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067435776:12345-192.168.203.37@tcp:4:dd.500 Request processed in 18960us (44086us total) trans 4294967848 rc 0/0 00000100:00100000:2.0:1789762864.223531:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 530 00000100:00100000:2.0:1789762864.223544:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337451468372 time 337451585393 state 0x2 ref 0 00000001:00080000:2.0:1789762864.223596:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967848 is committed 00000100:00100000:1.0:1789762864.226769:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762864.226828:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067435904 00000100:00100000:3.0:1789762864.226839:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067435904 00000100:00100000:3.0:1789762864.226891:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337451468372 time 337451585393 state 0x0 ref 1 00000100:00100000:3.0:1789762864.226904:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337501585393 time 337451585393 state 0x1 ref 1 00000100:00100000:0.0:1789762864.254671:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337501585393 time 337501726968 state 0x0 ref 1 00000100:00100000:0.0:1789762864.254717:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067435904:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762864.254801:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 39845888, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x13007 00010000:00010000:0.0:1789762864.254817:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176390 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.254844:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:0.0:1789762864.254852:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762864.254862:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176390 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.254881:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176390 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762864.270295:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 39845888 -> 40894464 00002000:00010000:0.0:1789762864.270323:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 77831 -> 79879 00000001:00080000:2.0:1789762864.270338:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967849 is committed 00010000:00010000:0.0:1789762864.270339:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176390 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.270360:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176390 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.270385:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176390 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.270391:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762864.270395:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176390 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.270402:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176390 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.270418:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176390 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.270425:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176390 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762864.270484:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067435904:12345-192.168.203.37@tcp:4:dd.500 Request processed in 15788us (43722us total) trans 4294967849 rc 0/0 00000100:00100000:0.0:1789762864.270503:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 531 00000100:00100000:0.0:1789762864.270512:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337501585393 time 337501726968 state 0x2 ref 0 00000100:00100000:1.0:1789762864.273263:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762864.273348:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067436032 00000100:00100000:2.0:1789762864.273359:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067436032 00000100:00100000:2.0:1789762864.273436:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337501585393 time 337501726968 state 0x0 ref 1 00000100:00100000:2.0:1789762864.273449:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337551726968 time 337501726968 state 0x1 ref 1 00000100:00100000:3.0:1789762864.304768:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337551726968 time 337551828041 state 0x0 ref 1 00000100:00100000:3.0:1789762864.304795:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067436032:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762864.304866:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 40894464, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x13807 00010000:00010000:3.0:1789762864.304878:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176397 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.304909:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:3.0:1789762864.304915:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762864.304949:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.304980:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762864.324908:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967850 is committed 00002000:00010000:3.0:1789762864.325031:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 40894464 -> 41943040 00002000:00010000:3.0:1789762864.325039:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 79879 -> 81927 00010000:00010000:3.0:1789762864.325052:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.325063:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176397 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.325070:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176397 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.325075:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762864.325079:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176397 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.325088:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176397 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.325106:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176397 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.325111:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176397 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762864.325194:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067436032:12345-192.168.203.37@tcp:4:dd.500 Request processed in 20404us (51929us total) trans 4294967850 rc 0/0 00000100:00100000:3.0:1789762864.325203:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 532 00000100:00100000:3.0:1789762864.325213:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337551726968 time 337551828041 state 0x2 ref 0 00000100:00100000:2.0:1789762864.329109:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762864.329153:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067436160 00000100:00100000:3.0:1789762864.329165:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067436160 00000100:00100000:3.0:1789762864.329225:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337551726968 time 337551828041 state 0x0 ref 1 00000100:00100000:3.0:1789762864.329243:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337601828041 time 337551828041 state 0x1 ref 1 00000100:00100000:0.0:1789762864.354966:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337601828041 time 337601993352 state 0x0 ref 1 00000100:00100000:0.0:1789762864.354999:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9254000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067436160:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762864.355062:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 41943040, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x14007 00010000:00010000:0.0:1789762864.355074:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17639e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.355095:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:0.0:1789762864.355101:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762864.355110:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17639e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.355128:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17639e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762864.378083:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967851 is committed 00002000:00010000:3.0:1789762864.378146:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 41943040 -> 42991616 00002000:00010000:3.0:1789762864.378154:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 81927 -> 83975 00010000:00010000:3.0:1789762864.378173:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17639e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.378188:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17639e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.378200:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17639e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.378210:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762864.378216:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17639e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.378240:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17639e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.378270:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17639e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.378280:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17639e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762864.378425:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9254000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067436160:12345-192.168.203.37@tcp:4:dd.500 Request processed in 23429us (49308us total) trans 4294967851 rc 0/0 00000100:00100000:3.0:1789762864.378440:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 533 00000100:00100000:3.0:1789762864.378459:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337601828041 time 337601993352 state 0x2 ref 0 00000100:00100000:2.0:1789762864.382778:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762864.382879:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067436288 00000100:00100000:0.0:1789762864.382898:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067436288 00000100:00100000:0.0:1789762864.383024:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337601828041 time 337601993352 state 0x0 ref 1 00000100:00100000:0.0:1789762864.383046:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337651993352 time 337601993352 state 0x1 ref 1 00000100:00100000:0.0:1789762864.405051:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337651993352 time 337652116386 state 0x0 ref 1 00000100:00100000:0.0:1789762864.405078:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9256300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067436288:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762864.405140:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 42991616, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x14807 00010000:00010000:0.0:1789762864.405152:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.405174:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:0.0:1789762864.405180:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762864.405188:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.405206:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762864.424441:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967852 is committed 00002000:00010000:2.0:1789762864.424462:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 42991616 -> 44040192 00002000:00010000:2.0:1789762864.424470:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 83975 -> 86023 00010000:00010000:2.0:1789762864.424491:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.424505:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.424513:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.424522:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762864.424528:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.424538:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.424562:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.424582:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762864.424691:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9256300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067436288:12345-192.168.203.37@tcp:4:dd.500 Request processed in 19617us (41911us total) trans 4294967852 rc 0/0 00000100:00100000:2.0:1789762864.424706:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 534 00000100:00100000:2.0:1789762864.424722:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337651993352 time 337652116386 state 0x2 ref 0 00000100:00100000:2.0:1789762864.428431:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762864.428478:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067436416 00000100:00100000:3.0:1789762864.428489:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067436416 00000100:00100000:3.0:1789762864.428541:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337651993352 time 337652116386 state 0x0 ref 1 00000100:00100000:3.0:1789762864.428556:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337702116386 time 337652116386 state 0x1 ref 1 00000100:00100000:0.0:1789762864.455217:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337702116386 time 337702271700 state 0x0 ref 1 00000100:00100000:0.0:1789762864.455257:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9257b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067436416:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762864.455347:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 44040192, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x15007 00010000:00010000:0.0:1789762864.455384:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.455408:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:0.0:1789762864.455415:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762864.455445:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.455465:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762864.474403:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967853 is committed 00002000:00010000:2.0:1789762864.474499:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 44040192 -> 45088768 00002000:00010000:2.0:1789762864.474506:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 86023 -> 88071 00010000:00010000:2.0:1789762864.474522:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.474535:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.474542:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.474547:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762864.474553:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.474562:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.474607:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.474614:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762864.474728:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9257b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067436416:12345-192.168.203.37@tcp:4:dd.500 Request processed in 19484us (46291us total) trans 4294967853 rc 0/0 00000100:00100000:2.0:1789762864.474742:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 535 00000100:00100000:2.0:1789762864.474756:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337702116386 time 337702271700 state 0x2 ref 0 00000100:00100000:1.0:1789762864.478380:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762864.478443:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067436544 00000100:00100000:3.0:1789762864.478472:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067436544 00000100:00100000:3.0:1789762864.478533:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337702116386 time 337702271700 state 0x0 ref 1 00000100:00100000:3.0:1789762864.478560:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337752271700 time 337702271700 state 0x1 ref 1 00000100:00100000:2.0:1789762864.505280:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337752271700 time 337752346276 state 0x0 ref 1 00000100:00100000:2.0:1789762864.505303:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067436544:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762864.505371:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 45088768, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x15807 00010000:00010000:2.0:1789762864.505381:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.505397:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:2.0:1789762864.505401:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762864.505407:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.505421:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762864.517398:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967854 is committed 00002000:00010000:2.0:1789762864.517444:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 45088768 -> 46137344 00002000:00010000:2.0:1789762864.517451:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 88071 -> 90119 00010000:00010000:2.0:1789762864.517463:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.517474:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.517484:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.517493:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762864.517498:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.517508:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.517528:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.517536:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762864.517620:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067436544:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12322us (39260us total) trans 4294967854 rc 0/0 00000100:00100000:2.0:1789762864.517641:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 536 00000100:00100000:2.0:1789762864.517655:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337752271700 time 337752346276 state 0x2 ref 0 00000100:00100000:1.0:1789762864.519647:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762864.519719:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067436672 00000100:00100000:2.0:1789762864.519729:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067436672 00000100:00100000:2.0:1789762864.519768:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337752271700 time 337752346276 state 0x0 ref 1 00000100:00100000:2.0:1789762864.519780:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337802346276 time 337752346276 state 0x1 ref 1 00000100:00100000:2.0:1789762864.555344:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337802346276 time 337802408997 state 0x0 ref 1 00000100:00100000:2.0:1789762864.555386:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a09c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067436672:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762864.555429:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 46137344, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x16007 00010000:00010000:2.0:1789762864.555437:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.555451:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:2.0:1789762864.555455:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762864.555460:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.555471:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762864.567236:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967855 is committed 00002000:00010000:2.0:1789762864.567288:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 46137344 -> 47185920 00002000:00010000:2.0:1789762864.567294:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 90119 -> 92167 00010000:00010000:2.0:1789762864.567305:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.567316:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.567326:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.567335:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762864.567340:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.567352:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.567390:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.567397:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1763ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762864.567464:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a09c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067436672:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12103us (47818us total) trans 4294967855 rc 0/0 00000100:00100000:2.0:1789762864.567481:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 537 00000100:00100000:2.0:1789762864.567490:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337802346276 time 337802408997 state 0x2 ref 0 00000100:00100000:2.0:1789762864.569499:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762864.569553:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067436800 00000100:00100000:0.0:1789762864.569561:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067436800 00000100:00100000:0.0:1789762864.569608:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337802346276 time 337802408997 state 0x0 ref 1 00000100:00100000:0.0:1789762864.569620:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337852408997 time 337802408997 state 0x1 ref 1 00000100:00100000:0.0:1789762864.605376:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337852408997 time 337852439804 state 0x0 ref 1 00000100:00100000:0.0:1789762864.605393:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067436800:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762864.605434:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 47185920, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x16807 00010000:00010000:0.0:1789762864.605441:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1763c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.605455:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:0.0:1789762864.605458:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762864.605463:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1763c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.605473:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1763c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762864.615865:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967856 is committed 00002000:00010000:2.0:1789762864.615903:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 47185920 -> 48234496 00002000:00010000:2.0:1789762864.615911:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 92167 -> 94215 00010000:00010000:2.0:1789762864.615941:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1763c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.615952:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1763c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.615959:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1763c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.615964:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762864.615968:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1763c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.615975:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1763c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.615993:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1763c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.615998:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1763c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762864.616088:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067436800:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10695us (46586us total) trans 4294967856 rc 0/0 00000100:00100000:2.0:1789762864.616097:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 538 00000100:00100000:2.0:1789762864.616107:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337852408997 time 337852439804 state 0x2 ref 0 00000100:00100000:0.0:1789762864.618016:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762864.618053:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067436928 00000100:00100000:2.0:1789762864.618062:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067436928 00000100:00100000:2.0:1789762864.618106:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337852408997 time 337852439804 state 0x0 ref 1 00000100:00100000:2.0:1789762864.618118:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337902439804 time 337852439804 state 0x1 ref 1 00000004:00080000:0.0:1789762864.640094:0:9022:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 337887173906ns, next at 338887170361ns, rc = 0 00000004:00080000:1.0:1789762864.640096:0:9018:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 337887176591ns, next at 338887174414ns, rc = 0 00000100:00100000:3.0:1789762864.640101:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074698496:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762864.640129:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8a9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074698624:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762864.640173:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762864.640216:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074698368:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762864.640269:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762864.640272:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d275ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074699008:0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.640299:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d275a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074698752:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.640320:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d275aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074698880:0@lo:400:kworker.0 00000100:00100000:3.0:1789762864.640320:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762864.640356:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074699520:0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.640395:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762864.640396:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762864.640403:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762864.640476:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074699264:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.640479:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074699392:0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.640485:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074699136:0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.640589:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762864.640599:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074698496 00000100:00100000:1.0:1789762864.640601:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074698368 00000100:00100000:1.0:1789762864.640609:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074698368 00000100:00100000:0.0:1789762864.640610:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074698496 00000100:00100000:3.0:1789762864.640614:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762864.640630:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074699776:0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.640655:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074698624 00000100:00100000:1.0:1789762864.640659:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074698624 00000100:00100000:3.0:1789762864.640664:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074700032:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.640674:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:2.0:1789762864.640683:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762864.640696:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074698368:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762864.640707:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00002000:00000024:0.0:1789762864.640721:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526188544 pending 0 free 3833593856 avail 3792699392 00000100:00100000:1.0:1789762864.640725:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074698496:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762864.640725:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762864.640732:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1abbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074700288:0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789762864.640746:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526188544 pending 0 free 3833593856 avail 3792699392 00000100:00100000:2.0:1789762864.640793:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762864.640795:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074698368:12345-0@lo:13:osp-pre-0-1.0 Request processed in 96us (517us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762864.640809:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074698496:12345-0@lo:13:osp-pre-0-0.0 Request processed in 91us (635us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762864.640811:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:2.0:1789762864.640823:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789762864.640824:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00100000:2.0:1789762864.640836:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074699904:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.640847:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074700288 00000100:00100000:0.0:1789762864.640856:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074700288 00000100:00100000:1.0:1789762864.640856:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:2.0:1789762864.640884:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762864.640885:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074698624:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762864.640895:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 201 00000100:00100000:0.0:1789762864.640914:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074700288:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789762864.640945:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 96927744 pending 25559040 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789762864.640962:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074699008 00002000:00000024:0.0:1789762864.640963:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 96927744 pending 25559040 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789762864.640967:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074699008 00000100:00100000:2.0:1789762864.640978:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074698880 00000100:00100000:1.0:1789762864.640993:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074698624:12345-0@lo:13:osp-pre-1-1.0 Request processed in 124us (606us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762864.640996:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074698880 00000100:00100000:3.0:1789762864.641008:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074699264 00000100:00100000:1.0:1789762864.641009:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00100000:3.0:1789762864.641016:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074699264 00000100:00100000:0.0:1789762864.641025:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074700288:12345-0@lo:13:osp-pre-1-0.0 Request processed in 117us (237us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762864.641044:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 201 00000100:00100000:2.0:1789762864.641059:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074699520 00000100:00100000:2.0:1789762864.641064:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074699520 00000100:00080000:1.0:1789762864.641066:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789762864.641071:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762864.641071:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 327 00000100:00080000:1.0:1789762864.641077:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762864.641090:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:3.0:1789762864.641090:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074699008:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.641103:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074699264:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.641112:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074699776 00000100:00100000:1.0:1789762864.641118:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074699776 00000100:00100000:2.0:1789762864.641119:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 482 00000100:00100000:2.0:1789762864.641143:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074698880:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762864.641152:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074699008:12345-0@lo:400:kworker.0 Request processed in 67us (831us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762864.641153:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074699264:12345-0@lo:400:kworker.0 Request processed in 48us (563us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762864.641166:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00100000:3.0:1789762864.641174:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 327 00000100:00100000:1.0:1789762864.641175:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 483 00000100:00100000:1.0:1789762864.641190:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074699520:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762864.641204:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074698752 00000100:00100000:0.0:1789762864.641209:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 484 00000100:00100000:3.0:1789762864.641211:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074698752 00000100:00100000:2.0:1789762864.641218:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074698880:12345-0@lo:400:kworker.0 Request processed in 87us (829us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762864.641230:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074699776:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.641233:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 482 00000100:00100000:1.0:1789762864.641245:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074699520:12345-0@lo:400:kworker.0 Request processed in 54us (673us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762864.641258:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 483 00000100:00100000:3.0:1789762864.641262:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 358 00000100:00100000:3.0:1789762864.641285:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1551c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074698752:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.641287:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074700032 00000100:00080000:1.0:1789762864.641288:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762864.641297:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074700032 00000100:00100000:0.0:1789762864.641299:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074699776:12345-0@lo:400:kworker.0 Request processed in 79us (621us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762864.641313:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 484 00000100:00100000:3.0:1789762864.641341:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1551c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074698752:12345-0@lo:400:kworker.0 Request processed in 61us (955us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762864.641346:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074699904 00000100:00100000:2.0:1789762864.641350:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074699904 00000100:00100000:1.0:1789762864.641356:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1abbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074700288:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762864.641356:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 358 00000100:00100000:0.0:1789762864.641375:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:0.0:1789762864.641392:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074700032:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.641395:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d275a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074698752:0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.641405:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:2.0:1789762864.641415:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074699904:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.641421:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2759500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074699264:0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.641445:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074699776:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.641463:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074700032:12345-0@lo:400:kworker.0 Request processed in 72us (743us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762864.641476:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:2.0:1789762864.641482:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074699904:12345-0@lo:400:kworker.0 Request processed in 68us (609us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762864.641492:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00100000:1.0:1789762864.641516:0:3424:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294967826 gen 2 00000004:00080000:1.0:1789762864.641521:0:3424:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c36d2a13480, transno 4294967826 00000100:00100000:1.0:1789762864.641535:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074699904:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.641549:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762864.641594:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074699648:0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.641608:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074699136 00000100:00100000:2.0:1789762864.641616:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074699136 00000100:00100000:0.0:1789762864.641636:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762864.641659:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074699648 00000100:00100000:2.0:1789762864.641665:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074699648 00000100:00100000:0.0:1789762864.641675:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074700160:0@lo:400:kworker.0 00000100:00100000:3.0:1789762864.641683:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 485 00000100:00100000:3.0:1789762864.641698:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074699136:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.641714:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 486 00000100:00100000:0.0:1789762864.641727:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762864.641733:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074699648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762864.641748:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074699136:12345-0@lo:400:kworker.0 Request processed in 56us (201us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762864.641758:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 485 00000100:00100000:0.0:1789762864.641794:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8a9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074698624:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762864.641803:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074699648:12345-0@lo:400:kworker.0 Request processed in 73us (167us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762864.641813:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 486 00000100:00100000:3.0:1789762864.641829:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762864.641842:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074699392 00000100:00100000:0.0:1789762864.641844:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2758000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074699136:0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.641847:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074699392 00000100:00100000:0.0:1789762864.641887:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074699648:0@lo:400:kworker.0 00000100:00100000:3.0:1789762864.641890:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074698368:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762864.641897:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 487 00000100:00100000:3.0:1789762864.641909:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d275aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074698880:0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.641951:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074699392:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.641956:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074700160 00000100:00100000:0.0:1789762864.641963:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074700160 00000100:00100000:2.0:1789762864.642025:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074699392:12345-0@lo:400:kworker.0 Request processed in 111us (201us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762864.642035:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:2.0:1789762864.642044:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 487 00000100:00100000:3.0:1789762864.642044:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074699392:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.642050:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074700160:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789762864.642112:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762864.642113:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d275ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074699008:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.642129:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074700160:12345-0@lo:400:kworker.0 Request processed in 74us (403us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762864.642145:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074699520:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.642146:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00100000:2.0:1789762864.642177:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074700032:0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.642195:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074700160:0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.642215:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074698496:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762864.655439:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337902439804 time 337902510447 state 0x0 ref 1 00000100:00100000:3.0:1789762864.655452:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067436928:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762864.655483:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 48234496, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x17007 00010000:00010000:3.0:1789762864.655489:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f1763c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.655502:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:3.0:1789762864.655505:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762864.655509:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f1763c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.655530:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f1763c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762864.666857:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967857 is committed 00002000:00010000:0.0:1789762864.666860:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 48234496 -> 49283072 00002000:00010000:0.0:1789762864.666866:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 94215 -> 96265 00010000:00010000:0.0:1789762864.666877:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f1763c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.666890:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f1763c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.666900:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f1763c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.666908:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762864.666913:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f1763c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.666944:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f1763c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.666976:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f1763c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.666985:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f1763c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762864.667046:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067436928:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11594us (49031us total) trans 4294967857 rc 0/0 00000100:00100000:0.0:1789762864.667052:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 539 00000100:00100000:0.0:1789762864.667060:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337902439804 time 337902510447 state 0x2 ref 0 00000100:00100000:2.0:1789762864.669118:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762864.669166:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067437056 00000100:00100000:3.0:1789762864.669173:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067437056 00000100:00100000:3.0:1789762864.669206:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337902439804 time 337902510447 state 0x0 ref 1 00000100:00100000:3.0:1789762864.669216:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337952510447 time 337902510447 state 0x1 ref 1 00000100:00100000:3.0:1789762864.705479:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337952510447 time 337952546978 state 0x0 ref 1 00000100:00100000:3.0:1789762864.705499:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067437056:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762864.705542:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 49283072, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x17809 00010000:00010000:3.0:1789762864.705550:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f1763cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.705562:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=337s v=(0 0) 00010000:00010000:3.0:1789762864.705566:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762864.705570:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f1763cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.705580:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f1763cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762864.717019:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967858 is committed 00002000:00010000:2.0:1789762864.717043:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 49283072 -> 50331648 00002000:00010000:2.0:1789762864.717054:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 96265 -> 98313 00010000:00010000:2.0:1789762864.717071:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f1763cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.717083:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f1763cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.717090:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f1763cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.717096:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762864.717100:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f1763cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.717107:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f1763cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.717136:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f1763cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.717143:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d20014d/0xdf5a7ac59f1763cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762864.717224:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067437056:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11727us (48103us total) trans 4294967858 rc 0/0 00000100:00100000:2.0:1789762864.717233:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 540 00000100:00100000:2.0:1789762864.717244:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337952510447 time 337952546978 state 0x2 ref 0 00000100:00100000:2.0:1789762864.719457:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762864.719509:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067437184 00000100:00100000:1.0:1789762864.719517:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067437184 00000100:00100000:1.0:1789762864.719557:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 337952510447 time 337952546978 state 0x0 ref 1 00000100:00100000:1.0:1789762864.719568:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338002546978 time 337952546978 state 0x1 ref 1 00000100:00100000:0.0:1789762864.755552:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338002546978 time 338002621839 state 0x0 ref 1 00000100:00100000:0.0:1789762864.755570:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067437184:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762864.755608:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 50331648, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x18009 00010000:00010000:0.0:1789762864.755615:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.755628:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:0.0:1789762864.755631:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762864.755635:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.755663:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762864.766727:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967859 is committed 00002000:00010000:2.0:1789762864.766744:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 50331648 -> 51380224 00002000:00010000:2.0:1789762864.766752:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 98313 -> 100361 00010000:00010000:2.0:1789762864.766767:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.766778:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.766785:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.766791:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762864.766795:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.766812:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.766831:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.766836:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f1763d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762864.766953:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067437184:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11353us (47461us total) trans 4294967859 rc 0/0 00000100:00100000:2.0:1789762864.766967:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 541 00000100:00100000:2.0:1789762864.766982:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338002546978 time 338002621839 state 0x2 ref 0 00010000:00010000:1.0:1789762864.768031:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762864.768879:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762864.768965:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067437312 00000100:00100000:0.0:1789762864.768974:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067437312 00000100:00100000:0.0:1789762864.769013:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338002546978 time 338002621839 state 0x0 ref 1 00000100:00100000:0.0:1789762864.769023:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338052621839 time 338002621839 state 0x1 ref 1 00000100:00100000:0.0:1789762864.805569:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338052621839 time 338052640906 state 0x0 ref 1 00000100:00100000:0.0:1789762864.805585:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067437312:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762864.805623:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 51380224, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x18809 00010000:00010000:0.0:1789762864.805629:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.805641:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:0.0:1789762864.805644:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762864.805649:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762864.805658:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762864.816616:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967860 is committed 00002000:00010000:3.0:1789762864.816631:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 51380224 -> 52428800 00002000:00010000:3.0:1789762864.816636:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 100361 -> 102409 00010000:00010000:3.0:1789762864.816647:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.816660:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.816670:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.816678:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762864.816683:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.816694:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.816713:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.816722:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1763dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762864.816782:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067437312:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11199us (47902us total) trans 4294967860 rc 0/0 00000100:00100000:3.0:1789762864.816800:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 542 00000100:00100000:3.0:1789762864.816811:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338052621839 time 338052640906 state 0x2 ref 0 00000100:00100000:1.0:1789762864.818747:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762864.818835:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067437440 00000100:00100000:2.0:1789762864.818848:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067437440 00000100:00100000:2.0:1789762864.818901:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338052621839 time 338052640906 state 0x0 ref 1 00000100:00100000:2.0:1789762864.818914:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338102640906 time 338052640906 state 0x1 ref 1 00000100:00100000:2.0:1789762864.855635:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338102640906 time 338102702114 state 0x0 ref 1 00000100:00100000:2.0:1789762864.855659:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067437440:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762864.855730:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 52428800, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x19009 00010000:00010000:2.0:1789762864.855738:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1763e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.855755:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:2.0:1789762864.855760:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762864.855767:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1763e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762864.855784:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1763e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762864.867976:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967861 is committed 00002000:00010000:3.0:1789762864.867981:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 52428800 -> 53477376 00002000:00010000:3.0:1789762864.867996:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 102409 -> 104457 00010000:00010000:3.0:1789762864.868009:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1763e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.868022:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1763e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.868033:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1763e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.868042:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762864.868047:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1763e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.868057:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1763e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.868075:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1763e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.868084:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f1763e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762864.868164:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067437440:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12512us (49415us total) trans 4294967861 rc 0/0 00000100:00100000:3.0:1789762864.868176:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 543 00000100:00100000:3.0:1789762864.868188:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338102640906 time 338102702114 state 0x2 ref 0 00000100:00100000:0.0:1789762864.870207:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762864.870254:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067437568 00000100:00100000:2.0:1789762864.870264:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067437568 00000100:00100000:2.0:1789762864.870324:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338102640906 time 338102702114 state 0x0 ref 1 00000100:00100000:2.0:1789762864.870340:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338152702114 time 338102702114 state 0x1 ref 1 00000100:00100000:3.0:1789762864.905758:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338152702114 time 338152824420 state 0x0 ref 1 00000100:00100000:3.0:1789762864.905782:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067437568:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762864.905829:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 53477376, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x19809 00010000:00010000:3.0:1789762864.905838:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f1763eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.905854:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:3.0:1789762864.905859:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762864.905865:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f1763eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.905880:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f1763eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762864.918802:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967862 is committed 00002000:00010000:3.0:1789762864.918822:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 53477376 -> 54525952 00002000:00010000:3.0:1789762864.918829:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 104457 -> 106505 00010000:00010000:3.0:1789762864.918840:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f1763eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.918860:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f1763eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.918871:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f1763eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.918879:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762864.918884:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f1763eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.918893:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f1763eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.918909:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f1763eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.918914:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f1763eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762864.918997:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067437568:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13222us (48789us total) trans 4294967862 rc 0/0 00000100:00100000:3.0:1789762864.919007:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 544 00000100:00100000:3.0:1789762864.919027:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338152702114 time 338152824420 state 0x2 ref 0 00000100:00100000:2.0:1789762864.921237:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762864.921279:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067437696 00000100:00100000:3.0:1789762864.921295:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067437696 00000100:00100000:3.0:1789762864.921336:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338152702114 time 338152824420 state 0x0 ref 1 00000100:00100000:3.0:1789762864.921347:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338202824420 time 338152824420 state 0x1 ref 1 00000100:00100000:1.0:1789762864.955850:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338202824420 time 338202914360 state 0x0 ref 1 00000100:00100000:1.0:1789762864.955880:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067437696:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762864.955954:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 54525952, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x1a009 00010000:00010000:1.0:1789762864.955962:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1763f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762864.955975:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:1.0:1789762864.955979:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762864.955984:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1763f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762864.955997:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1763f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762864.969384:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 54525952 -> 55574528 00002000:00010000:3.0:1789762864.969390:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 106505 -> 108553 00010000:00010000:3.0:1789762864.969411:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1763f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.969421:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1763f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.969431:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1763f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.969436:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762864.969443:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1763f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762864.969449:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967863 is committed 00010000:00010000:3.0:1789762864.969451:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1763f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.969468:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1763f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762864.969474:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1763f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762864.969537:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067437696:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13670us (48308us total) trans 4294967863 rc 0/0 00000100:00100000:3.0:1789762864.969545:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 545 00000100:00100000:3.0:1789762864.969554:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338202824420 time 338202914360 state 0x2 ref 0 00000100:00100000:2.0:1789762864.972023:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762864.972071:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067437824 00000100:00100000:3.0:1789762864.972080:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067437824 00000100:00100000:3.0:1789762864.972117:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338202824420 time 338202914360 state 0x0 ref 1 00000100:00100000:3.0:1789762864.972128:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338252914360 time 338202914360 state 0x1 ref 1 00000100:00100000:2.0:1789762865.005994:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338252914360 time 338253053074 state 0x0 ref 1 00000100:00100000:2.0:1789762865.006048:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca558a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067437824:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:1.0:1789762865.006106:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762865.006122:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 55574528, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x1a809 00010000:00010000:2.0:1789762865.006137:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f1763f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.006173:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:2.0:1789762865.006180:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762865.006190:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f1763f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.006211:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f1763f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762865.018789:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967864 is committed 00002000:00010000:3.0:1789762865.018823:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 55574528 -> 56623104 00002000:00010000:3.0:1789762865.018829:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 108553 -> 110601 00010000:00010000:3.0:1789762865.018839:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f1763f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.018848:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f1763f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.018854:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f1763f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.018859:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762865.018862:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f1763f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.018876:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f1763f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.018892:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f1763f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.018896:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f1763f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762865.018995:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca558a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067437824:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12974us (46975us total) trans 4294967864 rc 0/0 00000100:00100000:3.0:1789762865.019005:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 546 00000100:00100000:3.0:1789762865.019017:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338252914360 time 338253053074 state 0x2 ref 0 00000100:00100000:2.0:1789762865.021154:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.021198:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067437952 00000100:00100000:3.0:1789762865.021207:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067437952 00000100:00100000:3.0:1789762865.021243:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338252914360 time 338253053074 state 0x0 ref 1 00000100:00100000:3.0:1789762865.021253:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338303053074 time 338253053074 state 0x1 ref 1 00000100:00100000:2.0:1789762865.056107:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338303053074 time 338303150020 state 0x0 ref 1 00000100:00100000:2.0:1789762865.056145:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067437952:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762865.056218:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 56623104, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x1b009 00010000:00010000:2.0:1789762865.056231:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176400 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.056253:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:2.0:1789762865.056259:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762865.056268:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.056288:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762865.070466:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967865 is committed 00002000:00010000:3.0:1789762865.070494:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 56623104 -> 57671680 00002000:00010000:3.0:1789762865.070499:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 110601 -> 112649 00010000:00010000:3.0:1789762865.070510:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.070530:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176400 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.070537:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176400 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.070541:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762865.070545:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176400 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.070551:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176400 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.070566:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176400 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.070571:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081703a57/0xdf5a7ac59f176400 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762865.070637:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067437952:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14508us (49486us total) trans 4294967865 rc 0/0 00000100:00100000:3.0:1789762865.070646:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 547 00000100:00100000:3.0:1789762865.070656:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338303053074 time 338303150020 state 0x2 ref 0 00000100:00100000:2.0:1789762865.072882:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.072949:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067438080 00000100:00100000:3.0:1789762865.072965:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067438080 00000100:00100000:3.0:1789762865.073004:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338303053074 time 338303150020 state 0x0 ref 1 00000100:00100000:3.0:1789762865.073016:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338353150020 time 338303150020 state 0x1 ref 1 00000100:00100000:0.0:1789762865.106147:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338353150020 time 338353214973 state 0x0 ref 1 00000100:00100000:0.0:1789762865.106176:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca558380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067438080:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762865.106217:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 57671680, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x1b809 00010000:00010000:0.0:1789762865.106224:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f176407 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.106237:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:0.0:1789762865.106241:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762865.106245:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f176407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.106256:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f176407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762865.119193:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967866 is committed 00002000:00010000:3.0:1789762865.119228:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 57671680 -> 58720256 00002000:00010000:3.0:1789762865.119234:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 112649 -> 114697 00010000:00010000:3.0:1789762865.119249:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f176407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.119260:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f176407 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.119267:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f176407 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.119272:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762865.119275:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f176407 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.119282:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f176407 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.119298:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f176407 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.119302:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f28035ec/0xdf5a7ac59f176407 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762865.119397:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca558380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067438080:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13234us (46517us total) trans 4294967866 rc 0/0 00000100:00100000:3.0:1789762865.119406:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 548 00000100:00100000:3.0:1789762865.119415:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338353150020 time 338353214973 state 0x2 ref 0 00000100:00100000:2.0:1789762865.122128:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.122176:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067438208 00000100:00100000:3.0:1789762865.122185:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067438208 00000100:00100000:3.0:1789762865.122237:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338353150020 time 338353214973 state 0x0 ref 1 00000100:00100000:3.0:1789762865.122248:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338403214973 time 338353214973 state 0x1 ref 1 00000100:00100000:2.0:1789762865.156298:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338403214973 time 338403351771 state 0x0 ref 1 00000100:00100000:2.0:1789762865.156341:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067438208:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762865.156458:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 58720256, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x1c009 00010000:00010000:2.0:1789762865.156470:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f17640e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.156494:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:2.0:1789762865.156500:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762865.156508:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f17640e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.156528:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f17640e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762865.173295:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967867 is committed 00002000:00010000:3.0:1789762865.173303:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 58720256 -> 59768832 00002000:00010000:3.0:1789762865.173308:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 114697 -> 116745 00010000:00010000:3.0:1789762865.173319:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f17640e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.173330:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f17640e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.173337:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f17640e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.173342:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762865.173345:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f17640e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.173371:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f17640e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.173387:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f17640e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.173392:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f17640e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762865.173466:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067438208:12345-192.168.203.37@tcp:4:dd.500 Request processed in 17143us (51344us total) trans 4294967867 rc 0/0 00000100:00100000:3.0:1789762865.173477:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 549 00000100:00100000:3.0:1789762865.173491:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338403214973 time 338403351771 state 0x2 ref 0 00000100:00100000:1.0:1789762865.175896:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.175965:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067438336 00000100:00100000:3.0:1789762865.175974:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067438336 00000100:00100000:3.0:1789762865.176011:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338403214973 time 338403351771 state 0x0 ref 1 00000100:00100000:3.0:1789762865.176023:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338453351771 time 338403351771 state 0x1 ref 1 00000100:00100000:2.0:1789762865.206411:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338453351771 time 338453469571 state 0x0 ref 1 00000100:00100000:2.0:1789762865.206435:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067438336:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762865.206490:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 59768832, mtime: 0x6aad9d30, atime: 0x0, ctime: 0x6aad9d30, blocks: 0x1c809 00010000:00010000:2.0:1789762865.206498:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176415 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.206512:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:2.0:1789762865.206515:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762865.206532:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176415 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.206544:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176415 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762865.218576:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967868 is committed 00002000:00010000:3.0:1789762865.218608:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 59768832 -> 60817408 00002000:00010000:3.0:1789762865.218613:0:8057:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb mtime from disk: 1789762864 -> 1789762865 00002000:00010000:3.0:1789762865.218616:0:8057:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb ctime from disk: 1789762864 -> 1789762865 00002000:00010000:3.0:1789762865.218617:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 116745 -> 118793 00010000:00010000:3.0:1789762865.218638:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176415 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.218652:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176415 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.218662:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176415 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.218671:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762865.218677:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176415 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.218687:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176415 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.218704:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176415 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.218709:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012026acc/0xdf5a7ac59f176415 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762865.218786:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067438336:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12357us (42888us total) trans 4294967868 rc 0/0 00000100:00100000:3.0:1789762865.218795:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 550 00000100:00100000:3.0:1789762865.218805:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338453351771 time 338453469571 state 0x2 ref 0 00000100:00100000:0.0:1789762865.221201:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.221247:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067438464 00000100:00100000:3.0:1789762865.221256:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067438464 00000100:00100000:3.0:1789762865.221295:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338453351771 time 338453469571 state 0x0 ref 1 00000100:00100000:3.0:1789762865.221307:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338503469571 time 338453469571 state 0x1 ref 1 00000100:00100000:2.0:1789762865.256525:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338503469571 time 338503588122 state 0x0 ref 1 00000100:00100000:2.0:1789762865.256558:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067438464:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762865.256619:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 60817408, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x1d009 00010000:00010000:2.0:1789762865.256627:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f17641c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.256644:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:2.0:1789762865.256648:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762865.256655:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f17641c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.256668:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f17641c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762865.269336:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967869 is committed 00002000:00010000:0.0:1789762865.269379:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 60817408 -> 61865984 00002000:00010000:0.0:1789762865.269384:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 118793 -> 120841 00010000:00010000:0.0:1789762865.269395:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f17641c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.269416:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f17641c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.269422:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f17641c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.269427:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762865.269431:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f17641c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.269438:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f17641c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.269452:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f17641c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.269457:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5e557df/0xdf5a7ac59f17641c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762865.269528:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067438464:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12984us (48324us total) trans 4294967869 rc 0/0 00000100:00100000:0.0:1789762865.269536:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 551 00000100:00100000:0.0:1789762865.269545:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338503469571 time 338503588122 state 0x2 ref 0 00000100:00100000:0.0:1789762865.271816:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.271863:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067438592 00000100:00100000:3.0:1789762865.271871:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067438592 00000100:00100000:3.0:1789762865.271911:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338503469571 time 338503588122 state 0x0 ref 1 00000100:00100000:3.0:1789762865.271942:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338553588122 time 338503588122 state 0x1 ref 1 00000100:00100000:1.0:1789762865.306613:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338553588122 time 338553678827 state 0x0 ref 1 00000100:00100000:1.0:1789762865.306635:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067438592:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762865.306680:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 61865984, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x1d809 00010000:00010000:1.0:1789762865.306688:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f176423 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762865.306702:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:1.0:1789762865.306706:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762865.306711:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f176423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762865.306722:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f176423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762865.318985:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967870 is committed 00002000:00010000:0.0:1789762865.318988:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 61865984 -> 62914560 00002000:00010000:0.0:1789762865.318993:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 120841 -> 122889 00010000:00010000:0.0:1789762865.319004:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f176423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.319015:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f176423 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.319021:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f176423 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.319026:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998fa00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762865.319029:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f176423 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.319036:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f176423 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.319050:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f176423 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.319055:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f176423 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762865.319120:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067438592:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12490us (47302us total) trans 4294967870 rc 0/0 00000100:00100000:0.0:1789762865.319129:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 552 00000100:00100000:0.0:1789762865.319144:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338553588122 time 338553678827 state 0x2 ref 0 00000100:00100000:0.0:1789762865.321422:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.321465:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067438720 00000100:00100000:3.0:1789762865.321473:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067438720 00000100:00100000:3.0:1789762865.321513:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338553588122 time 338553678827 state 0x0 ref 1 00000100:00100000:3.0:1789762865.321524:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338603678827 time 338553678827 state 0x1 ref 1 00000100:00100000:0.0:1789762865.356730:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338603678827 time 338603798466 state 0x0 ref 1 00000100:00100000:0.0:1789762865.356751:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067438720:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762865.356797:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 62914560, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x1e009 00010000:00010000:0.0:1789762865.356803:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f17642a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.356816:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:0.0:1789762865.356820:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762865.356824:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f17642a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.356836:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f17642a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762865.369137:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967871 is committed 00002000:00010000:0.0:1789762865.369142:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 62914560 -> 63963136 00002000:00010000:0.0:1789762865.369148:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 122889 -> 124937 00010000:00010000:0.0:1789762865.369159:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f17642a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.369169:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f17642a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.369187:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f17642a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.369193:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762865.369197:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f17642a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.369203:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f17642a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.369217:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f17642a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.369222:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f17642a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762865.369277:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067438720:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12532us (47854us total) trans 4294967871 rc 0/0 00000100:00100000:0.0:1789762865.369288:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 553 00000100:00100000:0.0:1789762865.369296:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338603678827 time 338603798466 state 0x2 ref 0 00000100:00100000:2.0:1789762865.371435:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762865.371479:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067438848 00000100:00100000:0.0:1789762865.371487:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067438848 00000100:00100000:0.0:1789762865.371522:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338603678827 time 338603798466 state 0x0 ref 1 00000100:00100000:0.0:1789762865.371543:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338653798466 time 338603798466 state 0x1 ref 1 00000100:00100000:2.0:1789762865.406850:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338653798466 time 338653914830 state 0x0 ref 1 00000100:00100000:2.0:1789762865.406874:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca558e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067438848:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762865.406953:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 63963136, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x1e809 00010000:00010000:2.0:1789762865.406960:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f176431 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.406974:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:2.0:1789762865.406978:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762865.406982:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f176431 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.406993:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f176431 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762865.417729:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 63963136 -> 65011712 00002000:00010000:2.0:1789762865.417736:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 124937 -> 126985 00000001:00080000:0.0:1789762865.417737:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967872 is committed 00010000:00010000:2.0:1789762865.417747:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f176431 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.417757:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f176431 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.417764:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f176431 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.417769:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762865.417772:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f176431 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.417793:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f176431 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.417813:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f176431 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.417823:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f176431 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762865.417890:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca558e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067438848:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11022us (46460us total) trans 4294967872 rc 0/0 00000100:00100000:2.0:1789762865.417898:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 554 00000100:00100000:2.0:1789762865.417907:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338653798466 time 338653914830 state 0x2 ref 0 00000100:00100000:2.0:1789762865.420115:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762865.420195:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067438976 00000100:00100000:0.0:1789762865.420204:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067438976 00000100:00100000:0.0:1789762865.420242:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338653798466 time 338653914830 state 0x0 ref 1 00000100:00100000:0.0:1789762865.420253:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338703914830 time 338653914830 state 0x1 ref 1 00000100:00100000:0.0:1789762865.456905:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338703914830 time 338703974564 state 0x0 ref 1 00000100:00100000:0.0:1789762865.456939:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca559500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067438976:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762865.456981:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 65011712, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x1f009 00010000:00010000:0.0:1789762865.456988:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f176438 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.457001:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:0.0:1789762865.457005:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762865.457010:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f176438 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.457021:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f176438 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762865.467655:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967873 is committed 00002000:00010000:0.0:1789762865.467709:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 65011712 -> 66060288 00002000:00010000:0.0:1789762865.467714:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 126985 -> 129033 00010000:00010000:0.0:1789762865.467723:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f176438 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.467732:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f176438 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.467738:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f176438 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.467743:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762865.467746:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f176438 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.467753:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f176438 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.467767:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f176438 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.467772:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006fec596b/0xdf5a7ac59f176438 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762865.467825:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca559500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067438976:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10906us (47709us total) trans 4294967873 rc 0/0 00000100:00100000:0.0:1789762865.467840:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 555 00000100:00100000:0.0:1789762865.467848:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338703914830 time 338703974564 state 0x2 ref 0 00000100:00100000:2.0:1789762865.469947:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762865.469978:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067439104 00000100:00100000:0.0:1789762865.469986:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067439104 00000100:00100000:0.0:1789762865.470021:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338703914830 time 338703974564 state 0x0 ref 1 00000100:00100000:0.0:1789762865.470032:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338753974564 time 338703974564 state 0x1 ref 1 00000100:00100000:3.0:1789762865.507001:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338753974564 time 338754064812 state 0x0 ref 1 00000100:00100000:3.0:1789762865.507025:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca559880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067439104:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762865.507073:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 66060288, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x1f809 00010000:00010000:3.0:1789762865.507091:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17643f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.507104:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:3.0:1789762865.507108:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762865.507113:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17643f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.507124:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17643f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762865.518035:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967874 is committed 00002000:00010000:0.0:1789762865.518090:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 66060288 -> 67108864 00002000:00010000:0.0:1789762865.518094:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 129033 -> 131081 00010000:00010000:0.0:1789762865.518103:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17643f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.518112:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17643f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.518119:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17643f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.518124:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762865.518127:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17643f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.518133:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17643f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.518154:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17643f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.518160:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17643f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762865.518217:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca559880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067439104:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11198us (48272us total) trans 4294967874 rc 0/0 00000100:00100000:0.0:1789762865.518225:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 556 00000100:00100000:0.0:1789762865.518234:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338753974564 time 338754064812 state 0x2 ref 0 00000100:00100000:1.0:1789762865.520530:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762865.520586:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067439232 00000100:00100000:2.0:1789762865.520596:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067439232 00000100:00100000:2.0:1789762865.520635:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338753974564 time 338754064812 state 0x0 ref 1 00000100:00100000:2.0:1789762865.520646:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338804064812 time 338754064812 state 0x1 ref 1 00000100:00100000:3.0:1789762865.557105:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338804064812 time 338804172394 state 0x0 ref 1 00000100:00100000:3.0:1789762865.557127:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067439232:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762865.557173:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 67108864, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x20009 00010000:00010000:3.0:1789762865.557180:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176446 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.557193:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:3.0:1789762865.557196:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762865.557201:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.557212:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762865.568321:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 67108864 -> 68157440 00000001:00080000:0.0:1789762865.568328:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967875 is committed 00002000:00010000:2.0:1789762865.568328:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 131081 -> 133129 00010000:00010000:2.0:1789762865.568341:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.568373:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176446 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.568380:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176446 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.568385:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762865.568389:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176446 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.568395:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176446 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.568410:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176446 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.568415:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176446 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762865.568475:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067439232:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11354us (47944us total) trans 4294967875 rc 0/0 00000100:00100000:2.0:1789762865.568483:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 557 00000100:00100000:2.0:1789762865.568493:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338804064812 time 338804172394 state 0x2 ref 0 00000100:00100000:0.0:1789762865.570782:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.570830:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067439360 00000100:00100000:3.0:1789762865.570838:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067439360 00000100:00100000:3.0:1789762865.570877:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338804064812 time 338804172394 state 0x0 ref 1 00000100:00100000:3.0:1789762865.570887:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338854172394 time 338804172394 state 0x1 ref 1 00000100:00100000:2.0:1789762865.607190:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338854172394 time 338854259213 state 0x0 ref 1 00000100:00100000:2.0:1789762865.607219:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067439360:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762865.607262:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 68157440, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x20809 00010000:00010000:2.0:1789762865.607269:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f17644d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.607282:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:2.0:1789762865.607286:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762865.607290:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f17644d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.607301:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f17644d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762865.617869:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 68157440 -> 69206016 00002000:00010000:2.0:1789762865.617876:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 133129 -> 135177 00000001:00080000:3.0:1789762865.617882:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967876 is committed 00010000:00010000:2.0:1789762865.617887:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f17644d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.617902:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f17644d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.617911:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f17644d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.617933:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762865.617943:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f17644d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.617950:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f17644d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.617964:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f17644d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.617969:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000040b7cab/0xdf5a7ac59f17644d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762865.618020:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067439360:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10805us (47239us total) trans 4294967876 rc 0/0 00000100:00100000:2.0:1789762865.618028:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 558 00000100:00100000:2.0:1789762865.618036:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338854172394 time 338854259213 state 0x2 ref 0 00000100:00100000:1.0:1789762865.620272:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762865.620343:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067439488 00000100:00100000:0.0:1789762865.620363:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067439488 00000100:00100000:0.0:1789762865.620399:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338854172394 time 338854259213 state 0x0 ref 1 00000100:00100000:0.0:1789762865.620410:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338904259213 time 338854259213 state 0x1 ref 1 00000100:00100000:0.0:1789762865.657245:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338904259213 time 338904315251 state 0x0 ref 1 00000100:00100000:0.0:1789762865.657271:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067439488:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762865.657308:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 69206016, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x21009 00010000:00010000:0.0:1789762865.657325:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f176454 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.657337:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:0.0:1789762865.657340:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762865.657345:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f176454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.657367:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f176454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762865.667266:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967877 is committed 00002000:00010000:3.0:1789762865.667303:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 69206016 -> 70254592 00002000:00010000:3.0:1789762865.667307:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 135177 -> 137225 00010000:00010000:3.0:1789762865.667318:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f176454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.667328:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f176454 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.667335:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f176454 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.667339:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762865.667343:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f176454 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.667350:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f176454 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.667390:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f176454 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.667395:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f176454 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762865.667453:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067439488:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10193us (47179us total) trans 4294967877 rc 0/0 00000100:00100000:3.0:1789762865.667464:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 559 00000100:00100000:3.0:1789762865.667475:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338904259213 time 338904315251 state 0x2 ref 0 00000100:00100000:2.0:1789762865.669677:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.669711:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067439616 00000100:00100000:3.0:1789762865.669720:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067439616 00000100:00100000:3.0:1789762865.669755:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338904259213 time 338904315251 state 0x0 ref 1 00000100:00100000:3.0:1789762865.669765:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338954315251 time 338904315251 state 0x1 ref 1 00000100:00100000:0.0:1789762865.707321:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338954315251 time 338954390047 state 0x0 ref 1 00000100:00100000:0.0:1789762865.707341:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca559c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067439616:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762865.707396:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 70254592, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x21809 00010000:00010000:0.0:1789762865.707403:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17645b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.707416:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=338s v=(0 0) 00010000:00010000:0.0:1789762865.707419:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762865.707423:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17645b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.707434:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17645b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762865.717793:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967878 is committed 00002000:00010000:3.0:1789762865.717813:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 70254592 -> 71303168 00002000:00010000:3.0:1789762865.717820:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 137225 -> 139273 00010000:00010000:3.0:1789762865.717832:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17645b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.717846:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17645b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.717870:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17645b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.717878:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762865.717883:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17645b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.717893:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17645b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.717912:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17645b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.717941:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17645b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762865.717998:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca559c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067439616:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10661us (48319us total) trans 4294967878 rc 0/0 00000100:00100000:3.0:1789762865.718008:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 560 00000100:00100000:3.0:1789762865.718019:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338954315251 time 338954390047 state 0x2 ref 0 00000100:00100000:2.0:1789762865.720142:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762865.720186:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067439744 00000100:00100000:0.0:1789762865.720194:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067439744 00000100:00100000:0.0:1789762865.720229:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 338954315251 time 338954390047 state 0x0 ref 1 00000100:00100000:0.0:1789762865.720240:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339004390047 time 338954390047 state 0x1 ref 1 00000100:00100000:2.0:1789762865.757425:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339004390047 time 339004489234 state 0x0 ref 1 00000100:00100000:2.0:1789762865.757447:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067439744:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762865.757491:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 71303168, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x22009 00010000:00010000:2.0:1789762865.757498:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f176462 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.757525:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:2.0:1789762865.757528:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762865.757533:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f176462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.757545:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f176462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762865.768947:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967879 is committed 00002000:00010000:3.0:1789762865.768960:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 71303168 -> 72351744 00002000:00010000:3.0:1789762865.768965:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 139273 -> 141321 00010000:00010000:3.0:1789762865.768975:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f176462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.768995:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f176462 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.769001:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f176462 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.769006:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762865.769010:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f176462 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.769017:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f176462 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.769031:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f176462 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.769036:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f176462 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762865.769094:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067439744:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11654us (48951us total) trans 4294967879 rc 0/0 00000100:00100000:3.0:1789762865.769103:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 561 00000100:00100000:3.0:1789762865.769111:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339004390047 time 339004489234 state 0x2 ref 0 00000100:00100000:0.0:1789762865.771379:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.771412:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067439872 00000100:00100000:3.0:1789762865.771427:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067439872 00000100:00100000:3.0:1789762865.771464:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339004390047 time 339004489234 state 0x0 ref 1 00000100:00100000:3.0:1789762865.771476:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339054489234 time 339004489234 state 0x1 ref 1 00010000:00010000:3.0:1789762865.792057:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762865.807507:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339054489234 time 339054577550 state 0x0 ref 1 00000100:00100000:0.0:1789762865.807525:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067439872:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762865.807567:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 72351744, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x22809 00010000:00010000:0.0:1789762865.807574:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176469 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.807587:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:0.0:1789762865.807590:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762865.807595:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176469 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.807606:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176469 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762865.818094:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967880 is committed 00002000:00010000:3.0:1789762865.818108:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 72351744 -> 73400320 00002000:00010000:3.0:1789762865.818113:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 141321 -> 143369 00010000:00010000:3.0:1789762865.818123:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176469 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.818132:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176469 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.818138:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176469 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.818143:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762865.818147:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176469 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.818154:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176469 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.818168:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176469 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.818184:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176469 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762865.818239:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067439872:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10719us (46860us total) trans 4294967880 rc 0/0 00000100:00100000:3.0:1789762865.818247:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 562 00000100:00100000:3.0:1789762865.818256:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339054489234 time 339054577550 state 0x2 ref 0 00000100:00100000:1.0:1789762865.820413:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.820450:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067440000 00000100:00100000:3.0:1789762865.820458:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067440000 00000100:00100000:3.0:1789762865.820493:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339054489234 time 339054577550 state 0x0 ref 1 00000100:00100000:3.0:1789762865.820504:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339104577550 time 339054577550 state 0x1 ref 1 00000100:00100000:3.0:1789762865.857556:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339104577550 time 339104625149 state 0x0 ref 1 00000100:00100000:3.0:1789762865.857575:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067440000:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762865.857614:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 73400320, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x23009 00010000:00010000:3.0:1789762865.857621:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176470 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.857633:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:3.0:1789762865.857636:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762865.857651:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176470 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.857661:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176470 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762865.868991:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967881 is committed 00002000:00010000:3.0:1789762865.868998:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 73400320 -> 74448896 00002000:00010000:3.0:1789762865.869005:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 143369 -> 145417 00010000:00010000:3.0:1789762865.869016:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176470 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.869025:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176470 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.869032:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176470 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.869037:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762865.869041:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176470 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.869047:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176470 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.869061:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176470 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.869076:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176470 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762865.869148:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067440000:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11575us (48733us total) trans 4294967881 rc 0/0 00000100:00100000:3.0:1789762865.869156:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 563 00000100:00100000:3.0:1789762865.869166:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339104577550 time 339104625149 state 0x2 ref 0 00000100:00100000:2.0:1789762865.871379:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.871416:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067440128 00000100:00100000:3.0:1789762865.871425:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067440128 00000100:00100000:3.0:1789762865.871463:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339104577550 time 339104625149 state 0x0 ref 1 00000100:00100000:3.0:1789762865.871473:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339154625149 time 339104625149 state 0x1 ref 1 00000100:00100000:2.0:1789762865.907642:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339154625149 time 339154709335 state 0x0 ref 1 00000100:00100000:2.0:1789762865.907663:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca558700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067440128:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762865.907709:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 74448896, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x23809 00010000:00010000:2.0:1789762865.907715:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176477 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.907728:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:2.0:1789762865.907731:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762865.907736:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176477 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762865.907756:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176477 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762865.919048:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967882 is committed 00002000:00010000:3.0:1789762865.919066:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:3.0:1789762865.919071:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 145417 -> 147465 00010000:00010000:3.0:1789762865.919082:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176477 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.919091:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176477 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.919098:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176477 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.919113:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762865.919117:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176477 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.919123:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176477 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.919137:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176477 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.919143:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f176477 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762865.919204:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca558700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067440128:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11547us (47824us total) trans 4294967882 rc 0/0 00000100:00100000:3.0:1789762865.919213:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 564 00000100:00100000:3.0:1789762865.919223:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339154625149 time 339154709335 state 0x2 ref 0 00000100:00100000:1.0:1789762865.921469:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.921503:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067440256 00000100:00100000:3.0:1789762865.921512:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067440256 00000100:00100000:3.0:1789762865.921549:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339154625149 time 339154709335 state 0x0 ref 1 00000100:00100000:3.0:1789762865.921560:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339204709335 time 339154709335 state 0x1 ref 1 00000100:00100000:0.0:1789762865.957745:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339204709335 time 339204805971 state 0x0 ref 1 00000100:00100000:0.0:1789762865.957779:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067440256:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762865.957842:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 75497472, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x24009 00010000:00010000:0.0:1789762865.957851:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f17647e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.957868:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:0.0:1789762865.957872:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762865.957879:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f17647e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762865.957895:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f17647e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762865.969718:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967883 is committed 00002000:00010000:3.0:1789762865.969725:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 75497472 -> 76546048 00002000:00010000:3.0:1789762865.969735:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 147465 -> 149513 00010000:00010000:3.0:1789762865.969746:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f17647e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.969756:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f17647e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.969763:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f17647e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.969768:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762865.969771:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f17647e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.969778:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f17647e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.969793:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f17647e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762865.969798:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f17647e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762865.969864:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067440256:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12098us (48392us total) trans 4294967883 rc 0/0 00000100:00100000:3.0:1789762865.969872:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 565 00000100:00100000:3.0:1789762865.969882:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339204709335 time 339204805971 state 0x2 ref 0 00000100:00100000:0.0:1789762865.973247:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762865.973290:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067440384 00000100:00100000:3.0:1789762865.973301:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067440384 00000100:00100000:3.0:1789762865.973374:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339204709335 time 339204805971 state 0x0 ref 1 00000100:00100000:3.0:1789762865.973390:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339254805971 time 339204805971 state 0x1 ref 1 00000100:00100000:1.0:1789762866.007853:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339254805971 time 339254919022 state 0x0 ref 1 00000100:00100000:1.0:1789762866.007878:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067440384:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762866.007956:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 76546048, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x24809 00010000:00010000:1.0:1789762866.007965:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176485 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762866.007979:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:1.0:1789762866.007982:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762866.007987:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762866.007997:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762866.020128:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967884 is committed 00002000:00010000:3.0:1789762866.020163:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 76546048 -> 77594624 00002000:00010000:3.0:1789762866.020169:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 149513 -> 151561 00010000:00010000:3.0:1789762866.020182:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.020195:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176485 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.020218:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176485 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.020228:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.020233:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176485 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.020242:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176485 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.020262:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176485 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.020284:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176485 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.020373:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067440384:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12502us (47127us total) trans 4294967884 rc 0/0 00000100:00100000:3.0:1789762866.020384:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 566 00000100:00100000:3.0:1789762866.020397:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339254805971 time 339254919022 state 0x2 ref 0 00000100:00100000:0.0:1789762866.022650:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.022690:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067440512 00000100:00100000:3.0:1789762866.022700:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067440512 00000100:00100000:3.0:1789762866.022745:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339254805971 time 339254919022 state 0x0 ref 1 00000100:00100000:3.0:1789762866.022759:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339304919022 time 339254919022 state 0x1 ref 1 00000100:00100000:0.0:1789762866.057955:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339304919022 time 339305018699 state 0x0 ref 1 00000100:00100000:0.0:1789762866.057981:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067440512:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762866.058035:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 77594624, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x25009 00010000:00010000:0.0:1789762866.058044:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f17648c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.058090:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:0.0:1789762866.058095:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762866.058102:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f17648c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.058116:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f17648c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762866.070184:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967885 is committed 00002000:00010000:3.0:1789762866.070214:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:3.0:1789762866.070221:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 151561 -> 153609 00010000:00010000:3.0:1789762866.070233:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f17648c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.070246:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f17648c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.070256:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f17648c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.070278:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.070283:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f17648c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.070294:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f17648c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.070315:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f17648c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.070324:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f17648c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.070420:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067440512:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12444us (47767us total) trans 4294967885 rc 0/0 00000100:00100000:3.0:1789762866.070433:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 567 00000100:00100000:3.0:1789762866.070445:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339304919022 time 339305018699 state 0x2 ref 0 00000100:00100000:2.0:1789762866.072670:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.072710:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067440640 00000100:00100000:3.0:1789762866.072719:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067440640 00000100:00100000:3.0:1789762866.072761:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339304919022 time 339305018699 state 0x0 ref 1 00000100:00100000:3.0:1789762866.072775:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339355018699 time 339305018699 state 0x1 ref 1 00000100:00100000:2.0:1789762866.108058:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339355018699 time 339355125290 state 0x0 ref 1 00000100:00100000:2.0:1789762866.108079:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067440640:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762866.108122:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 78643200, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x25809 00010000:00010000:2.0:1789762866.108129:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176493 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.108141:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:2.0:1789762866.108145:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762866.108149:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.108160:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762866.119640:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967886 is committed 00002000:00010000:3.0:1789762866.119658:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:3.0:1789762866.119670:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 153609 -> 155657 00010000:00010000:3.0:1789762866.119680:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.119689:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176493 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.119695:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176493 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.119700:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.119704:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176493 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.119710:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176493 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.119725:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176493 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.119731:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176493 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.119790:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067440640:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11716us (47118us total) trans 4294967886 rc 0/0 00000100:00100000:3.0:1789762866.119798:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 568 00000100:00100000:3.0:1789762866.119807:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339355018699 time 339355125290 state 0x2 ref 0 00000100:00100000:1.0:1789762866.122030:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.122064:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067440768 00000100:00100000:3.0:1789762866.122072:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067440768 00000100:00100000:3.0:1789762866.122110:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339355018699 time 339355125290 state 0x0 ref 1 00000100:00100000:3.0:1789762866.122121:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339405125290 time 339355125290 state 0x1 ref 1 00000100:00100000:0.0:1789762866.158127:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339405125290 time 339405191561 state 0x0 ref 1 00000100:00100000:0.0:1789762866.158150:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067440768:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762866.158200:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 79691776, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x26009 00010000:00010000:0.0:1789762866.158207:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17649a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.158220:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:0.0:1789762866.158224:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762866.158230:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17649a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.158240:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17649a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762866.169821:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967887 is committed 00002000:00010000:3.0:1789762866.169830:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 79691776 -> 80740352 00002000:00010000:3.0:1789762866.169835:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 155657 -> 157705 00010000:00010000:3.0:1789762866.169847:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17649a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.169857:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17649a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.169863:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17649a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.169868:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.169872:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17649a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.169878:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17649a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.169892:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17649a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.169907:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f17649a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.169989:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067440768:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11845us (47956us total) trans 4294967887 rc 0/0 00000100:00100000:3.0:1789762866.169998:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 569 00000100:00100000:3.0:1789762866.170007:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339405125290 time 339405191561 state 0x2 ref 0 00000100:00100000:2.0:1789762866.172171:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.172203:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067440896 00000100:00100000:3.0:1789762866.172212:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067440896 00000100:00100000:3.0:1789762866.172247:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339405125290 time 339405191561 state 0x0 ref 1 00000100:00100000:3.0:1789762866.172257:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339455191561 time 339405191561 state 0x1 ref 1 00000100:00100000:0.0:1789762866.208224:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339455191561 time 339455288737 state 0x0 ref 1 00000100:00100000:0.0:1789762866.208263:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca559f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067440896:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762866.208313:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 80740352, mtime: 0x6aad9d31, atime: 0x0, ctime: 0x6aad9d31, blocks: 0x26809 00010000:00010000:0.0:1789762866.208323:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.208339:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:0.0:1789762866.208345:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762866.208367:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.208381:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762866.219611:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967888 is committed 00002000:00010000:3.0:1789762866.219631:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 80740352 -> 81788928 00002000:00010000:3.0:1789762866.219636:0:8059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb mtime from disk: 1789762865 -> 1789762866 00002000:00010000:3.0:1789762866.219638:0:8059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb ctime from disk: 1789762865 -> 1789762866 00002000:00010000:3.0:1789762866.219640:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 157705 -> 159753 00010000:00010000:3.0:1789762866.219649:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.219659:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.219666:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.219678:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.219682:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.219688:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.219703:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.219708:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.219767:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca559f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067440896:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11510us (47594us total) trans 4294967888 rc 0/0 00000100:00100000:3.0:1789762866.219774:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 570 00000100:00100000:3.0:1789762866.219784:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339455191561 time 339455288737 state 0x2 ref 0 00000100:00100000:1.0:1789762866.221975:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.222009:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067441024 00000100:00100000:3.0:1789762866.222017:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067441024 00000100:00100000:3.0:1789762866.222054:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339455191561 time 339455288737 state 0x0 ref 1 00000100:00100000:3.0:1789762866.222064:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339505288737 time 339455288737 state 0x1 ref 1 00000100:00100000:2.0:1789762866.258330:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339505288737 time 339505399045 state 0x0 ref 1 00000100:00100000:2.0:1789762866.258379:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067441024:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762866.258422:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 81788928, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x27009 00010000:00010000:2.0:1789762866.258429:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1764a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.258442:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:2.0:1789762866.258445:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762866.258450:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1764a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.258460:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1764a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762866.269512:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967889 is committed 00002000:00010000:3.0:1789762866.269522:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 81788928 -> 82837504 00002000:00010000:3.0:1789762866.269534:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 159753 -> 161801 00010000:00010000:3.0:1789762866.269544:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1764a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.269553:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1764a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.269563:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1764a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.269571:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.269576:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1764a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.269586:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1764a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.269605:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1764a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.269614:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6d2eb39/0xdf5a7ac59f1764a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.269670:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067441024:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11296us (47695us total) trans 4294967889 rc 0/0 00000100:00100000:3.0:1789762866.269678:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 571 00000100:00100000:3.0:1789762866.269686:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339505288737 time 339505399045 state 0x2 ref 0 00000100:00100000:0.0:1789762866.271784:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.271816:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067441152 00000100:00100000:3.0:1789762866.271824:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067441152 00000100:00100000:3.0:1789762866.271856:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339505288737 time 339505399045 state 0x0 ref 1 00000100:00100000:3.0:1789762866.271867:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339555399045 time 339505399045 state 0x1 ref 1 00000100:00100000:2.0:1789762866.308417:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339555399045 time 339555486327 state 0x0 ref 1 00000100:00100000:2.0:1789762866.308434:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067441152:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762866.308472:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 82837504, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x27809 00010000:00010000:2.0:1789762866.308478:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1764af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.308491:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:2.0:1789762866.308494:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762866.308498:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1764af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.308519:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1764af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762866.319767:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967890 is committed 00002000:00010000:3.0:1789762866.319772:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 82837504 -> 83886080 00002000:00010000:3.0:1789762866.319777:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 161801 -> 163849 00010000:00010000:3.0:1789762866.319786:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1764af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.319796:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1764af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.319803:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1764af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.319808:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.319811:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1764af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.319817:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1764af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.319831:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1764af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.319847:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075087339/0xdf5a7ac59f1764af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.319905:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067441152:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11474us (48121us total) trans 4294967890 rc 0/0 00000100:00100000:3.0:1789762866.319913:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 572 00000100:00100000:3.0:1789762866.319956:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339555399045 time 339555486327 state 0x2 ref 0 00000100:00100000:0.0:1789762866.322067:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.322100:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067441280 00000100:00100000:3.0:1789762866.322108:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067441280 00000100:00100000:3.0:1789762866.322146:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339555399045 time 339555486327 state 0x0 ref 1 00000100:00100000:3.0:1789762866.322156:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339605486327 time 339555486327 state 0x1 ref 1 00000100:00100000:1.0:1789762866.358516:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339605486327 time 339605585415 state 0x0 ref 1 00000100:00100000:1.0:1789762866.358536:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067441280:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762866.358575:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 83886080, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x28009 00010000:00010000:1.0:1789762866.358582:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1764b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762866.358594:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:1.0:1789762866.358598:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762866.358602:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1764b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762866.358612:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1764b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762866.369978:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 83886080 -> 84934656 00000001:00080000:2.0:1789762866.369985:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967891 is committed 00002000:00010000:3.0:1789762866.369985:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 163849 -> 165897 00010000:00010000:3.0:1789762866.369998:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1764b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.370008:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1764b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.370015:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1764b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.370030:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.370033:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1764b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.370040:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1764b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.370053:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1764b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.370059:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1764b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.370118:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067441280:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11587us (48050us total) trans 4294967891 rc 0/0 00000100:00100000:3.0:1789762866.370127:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 573 00000100:00100000:3.0:1789762866.370136:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339605486327 time 339605585415 state 0x2 ref 0 00000100:00100000:2.0:1789762866.372415:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.372449:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067441408 00000100:00100000:3.0:1789762866.372458:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067441408 00000100:00100000:3.0:1789762866.372493:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339605486327 time 339605585415 state 0x0 ref 1 00000100:00100000:3.0:1789762866.372503:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339655585415 time 339605585415 state 0x1 ref 1 00000100:00100000:0.0:1789762866.408591:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339655585415 time 339655658866 state 0x0 ref 1 00000100:00100000:0.0:1789762866.408610:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067441408:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762866.408659:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 84934656, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x28809 00010000:00010000:0.0:1789762866.408669:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd044021/0xdf5a7ac59f1764bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.408686:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:0.0:1789762866.408692:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762866.408709:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd044021/0xdf5a7ac59f1764bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.408720:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd044021/0xdf5a7ac59f1764bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762866.421467:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967892 is committed 00002000:00010000:0.0:1789762866.421485:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 84934656 -> 85983232 00002000:00010000:0.0:1789762866.421491:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 165897 -> 167945 00010000:00010000:0.0:1789762866.421502:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd044021/0xdf5a7ac59f1764bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.421513:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd044021/0xdf5a7ac59f1764bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.421520:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd044021/0xdf5a7ac59f1764bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.421525:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762866.421528:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd044021/0xdf5a7ac59f1764bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.421535:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd044021/0xdf5a7ac59f1764bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.421551:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd044021/0xdf5a7ac59f1764bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.421566:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd044021/0xdf5a7ac59f1764bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762866.421638:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067441408:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13030us (49219us total) trans 4294967892 rc 0/0 00000100:00100000:0.0:1789762866.421649:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 574 00000100:00100000:0.0:1789762866.421663:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339655585415 time 339655658866 state 0x2 ref 0 00000100:00100000:0.0:1789762866.424095:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.424128:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067441536 00000100:00100000:3.0:1789762866.424136:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067441536 00000100:00100000:3.0:1789762866.424174:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339655585415 time 339655658866 state 0x0 ref 1 00000100:00100000:3.0:1789762866.424186:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339705658866 time 339655658866 state 0x1 ref 1 00000100:00100000:2.0:1789762866.458682:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339705658866 time 339705749251 state 0x0 ref 1 00000100:00100000:2.0:1789762866.458701:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067441536:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762866.458740:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 85983232, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x29009 00010000:00010000:2.0:1789762866.458749:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f1764c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.458764:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:2.0:1789762866.458769:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762866.458777:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f1764c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.458791:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f1764c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762866.470080:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967893 is committed 00002000:00010000:3.0:1789762866.470088:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 85983232 -> 87031808 00002000:00010000:3.0:1789762866.470093:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 167945 -> 169993 00010000:00010000:3.0:1789762866.470102:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f1764c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.470112:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f1764c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.470119:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f1764c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.470124:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.470127:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f1764c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.470134:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f1764c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.470147:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f1764c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.470162:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f1764c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.470214:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067441536:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11516us (46117us total) trans 4294967893 rc 0/0 00000100:00100000:3.0:1789762866.470221:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 575 00000100:00100000:3.0:1789762866.470229:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339705658866 time 339705749251 state 0x2 ref 0 00000100:00100000:0.0:1789762866.472401:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.472436:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067441664 00000100:00100000:3.0:1789762866.472444:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067441664 00000100:00100000:3.0:1789762866.472479:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339705658866 time 339705749251 state 0x0 ref 1 00000100:00100000:3.0:1789762866.472489:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339755749251 time 339705749251 state 0x1 ref 1 00000100:00100000:0.0:1789762866.508915:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339755749251 time 339755964346 state 0x0 ref 1 00000100:00100000:0.0:1789762866.509025:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067441664:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762866.509126:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 87031808, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x29809 00010000:00010000:0.0:1789762866.509139:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1764cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.509161:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:0.0:1789762866.509168:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762866.509178:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1764cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.509198:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1764cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762866.520986:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967894 is committed 00002000:00010000:3.0:1789762866.521000:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 87031808 -> 88080384 00002000:00010000:3.0:1789762866.521006:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 169993 -> 172041 00010000:00010000:3.0:1789762866.521019:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1764cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.521033:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1764cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.521043:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1764cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.521065:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.521071:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1764cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.521081:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1764cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.521100:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1764cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.521107:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1764cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.521173:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067441664:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12168us (48768us total) trans 4294967894 rc 0/0 00000100:00100000:3.0:1789762866.521183:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 576 00000100:00100000:3.0:1789762866.521195:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339755749251 time 339755964346 state 0x2 ref 0 00000100:00100000:1.0:1789762866.523387:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762866.523439:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067441792 00000100:00100000:0.0:1789762866.523447:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067441792 00000100:00100000:0.0:1789762866.523485:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339755749251 time 339755964346 state 0x0 ref 1 00000100:00100000:0.0:1789762866.523495:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339805964346 time 339755964346 state 0x1 ref 1 00000100:00100000:0.0:1789762866.558951:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339805964346 time 339806023145 state 0x0 ref 1 00000100:00100000:0.0:1789762866.558967:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067441792:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762866.559001:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 88080384, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x2a009 00010000:00010000:0.0:1789762866.559008:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f1764d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.559019:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:0.0:1789762866.559023:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762866.559027:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f1764d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.559037:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f1764d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762866.570607:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967895 is committed 00002000:00010000:3.0:1789762866.570614:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 88080384 -> 89128960 00002000:00010000:3.0:1789762866.570619:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 172041 -> 174089 00010000:00010000:3.0:1789762866.570628:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f1764d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.570638:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f1764d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.570655:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f1764d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.570660:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.570663:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f1764d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.570669:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f1764d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.570682:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f1764d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.570688:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051f8e904/0xdf5a7ac59f1764d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.570747:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067441792:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11782us (47359us total) trans 4294967895 rc 0/0 00000100:00100000:3.0:1789762866.570754:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 577 00000100:00100000:3.0:1789762866.570762:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339805964346 time 339806023145 state 0x2 ref 0 00000100:00100000:1.0:1789762866.573063:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.573098:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067441920 00000100:00100000:3.0:1789762866.573105:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067441920 00000100:00100000:3.0:1789762866.573137:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339805964346 time 339806023145 state 0x0 ref 1 00000100:00100000:3.0:1789762866.573167:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339856023145 time 339806023145 state 0x1 ref 1 00000100:00100000:2.0:1789762866.609041:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339856023145 time 339856108226 state 0x0 ref 1 00000100:00100000:2.0:1789762866.609061:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067441920:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762866.609107:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 89128960, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x2a809 00010000:00010000:2.0:1789762866.609115:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b35bc55d/0xdf5a7ac59f1764d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.609129:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:2.0:1789762866.609134:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762866.609141:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b35bc55d/0xdf5a7ac59f1764d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.609155:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35bc55d/0xdf5a7ac59f1764d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762866.621494:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967896 is committed 00002000:00010000:3.0:1789762866.621526:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 89128960 -> 90177536 00002000:00010000:3.0:1789762866.621533:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 174089 -> 176137 00010000:00010000:3.0:1789762866.621543:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b35bc55d/0xdf5a7ac59f1764d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.621557:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b35bc55d/0xdf5a7ac59f1764d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.621567:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b35bc55d/0xdf5a7ac59f1764d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.621575:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.621579:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b35bc55d/0xdf5a7ac59f1764d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.621588:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b35bc55d/0xdf5a7ac59f1764d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.621601:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b35bc55d/0xdf5a7ac59f1764d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.621607:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35bc55d/0xdf5a7ac59f1764d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.621670:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067441920:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12613us (48605us total) trans 4294967896 rc 0/0 00000100:00100000:3.0:1789762866.621680:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 578 00000100:00100000:3.0:1789762866.621702:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339856023145 time 339856108226 state 0x2 ref 0 00000100:00100000:2.0:1789762866.623771:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.623806:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067442048 00000100:00100000:3.0:1789762866.623814:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067442048 00000100:00100000:3.0:1789762866.623857:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339856023145 time 339856108226 state 0x0 ref 1 00000100:00100000:3.0:1789762866.623867:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339906108226 time 339856108226 state 0x1 ref 1 00000100:00100000:3.0:1789762866.659094:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339906108226 time 339906164987 state 0x0 ref 1 00000100:00100000:3.0:1789762866.659111:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067442048:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762866.659146:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 90177536, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x2b009 00010000:00010000:3.0:1789762866.659152:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f1764e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.659164:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:3.0:1789762866.659167:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762866.659172:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f1764e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.659181:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f1764e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762866.670461:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967897 is committed 00002000:00010000:3.0:1789762866.670472:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 90177536 -> 91226112 00002000:00010000:3.0:1789762866.670477:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 176137 -> 178185 00010000:00010000:3.0:1789762866.670487:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f1764e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.670501:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f1764e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.670511:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f1764e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.670519:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.670524:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f1764e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.670533:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f1764e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.670550:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f1764e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.670559:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f1764e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.670619:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067442048:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11510us (46847us total) trans 4294967897 rc 0/0 00000100:00100000:3.0:1789762866.670637:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 579 00000100:00100000:3.0:1789762866.670646:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339906108226 time 339906164987 state 0x2 ref 0 00000100:00100000:1.0:1789762866.672641:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.672675:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067442176 00000100:00100000:3.0:1789762866.672683:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067442176 00000100:00100000:3.0:1789762866.672721:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339906108226 time 339906164987 state 0x0 ref 1 00000100:00100000:3.0:1789762866.672736:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339956164987 time 339906164987 state 0x1 ref 1 00000100:00100000:0.0:1789762866.688081:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789762866.688098:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d275aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074700416:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762866.688146:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762866.688182:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074700416 00000100:00100000:3.0:1789762866.688188:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074700416 00000100:00100000:3.0:1789762866.688235:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 329 00000100:00100000:3.0:1789762866.688246:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074700416:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789762866.688281:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762866.688439:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074700416:12345-0@lo:41:osp-pre-1-0.0 Request processed in 192us (287us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762866.688451:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 329 00000100:00100000:0.0:1789762866.688455:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d275aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074700416:0@lo:41:osp-pre-1-0.0 00000001:00080000:0.0:1789762866.701583:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967391 is committed 00000001:00080000:0.0:1789762866.701617:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967398 is committed 00000001:00080000:0.0:1789762866.701620:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967397 is committed 00000100:00100000:1.0:1789762866.709172:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339956164987 time 339956239784 state 0x0 ref 1 00000100:00100000:1.0:1789762866.709191:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067442176:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762866.709247:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 91226112, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x2b809 00010000:00010000:1.0:1789762866.709254:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1764e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762866.709267:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=339s v=(0 0) 00010000:00010000:1.0:1789762866.709271:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762866.709276:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1764e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762866.709287:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1764e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762866.720204:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 91226112 -> 92274688 00002000:00010000:1.0:1789762866.720211:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 178185 -> 180233 00010000:00010000:1.0:1789762866.720224:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1764e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762866.720245:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1764e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762866.720251:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967898 is committed 00010000:00010000:1.0:1789762866.720255:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1764e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762866.720264:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762866.720267:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1764e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762866.720275:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1764e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762866.720290:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1764e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762866.720296:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1764e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762866.720375:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067442176:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11187us (47732us total) trans 4294967898 rc 0/0 00000100:00100000:1.0:1789762866.720383:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 580 00000100:00100000:1.0:1789762866.720393:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339956164987 time 339956239784 state 0x2 ref 0 00000100:00100000:2.0:1789762866.722708:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.722749:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067442304 00000100:00100000:3.0:1789762866.722758:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067442304 00000100:00100000:3.0:1789762866.722798:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 339956164987 time 339956239784 state 0x0 ref 1 00000100:00100000:3.0:1789762866.722809:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340006239784 time 339956239784 state 0x1 ref 1 00000100:00100000:0.0:1789762866.759277:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340006239784 time 340006346985 state 0x0 ref 1 00000100:00100000:0.0:1789762866.759296:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca558000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067442304:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762866.759338:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 92274688, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x2c009 00010000:00010000:0.0:1789762866.759359:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.759371:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:0.0:1789762866.759375:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762866.759380:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.759392:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762866.769910:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967899 is committed 00002000:00010000:3.0:1789762866.769937:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 92274688 -> 93323264 00002000:00010000:3.0:1789762866.769943:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 180233 -> 182281 00010000:00010000:3.0:1789762866.769952:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.769972:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.769979:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.769984:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.769988:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.769993:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.770007:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.770012:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c972f9/0xdf5a7ac59f1764ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.770065:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca558000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067442304:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10774us (47356us total) trans 4294967899 rc 0/0 00000100:00100000:3.0:1789762866.770072:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 581 00000100:00100000:3.0:1789762866.770084:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340006239784 time 340006346985 state 0x2 ref 0 00000100:00100000:2.0:1789762866.772100:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.772136:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067442432 00000100:00100000:3.0:1789762866.772149:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067442432 00000100:00100000:3.0:1789762866.772181:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340006239784 time 340006346985 state 0x0 ref 1 00000100:00100000:3.0:1789762866.772191:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340056346985 time 340006346985 state 0x1 ref 1 00000100:00100000:2.0:1789762866.809390:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340056346985 time 340056462329 state 0x0 ref 1 00000100:00100000:2.0:1789762866.809406:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067442432:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762866.809437:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 93323264, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x2c809 00010000:00010000:2.0:1789762866.809443:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f1764f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.809455:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:2.0:1789762866.809466:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762866.809471:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f1764f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.809481:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f1764f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.815988:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789762866.819701:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 93323264 -> 94371840 00002000:00010000:2.0:1789762866.819706:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 182281 -> 184329 00000001:00080000:0.0:1789762866.819711:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967900 is committed 00010000:00010000:2.0:1789762866.819714:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f1764f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.819724:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f1764f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.819731:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f1764f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.819736:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762866.819739:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f1764f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.819745:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f1764f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.819757:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f1764f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.819769:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba1f7444/0xdf5a7ac59f1764f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762866.819823:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067442432:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10419us (47721us total) trans 4294967900 rc 0/0 00000100:00100000:2.0:1789762866.819834:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 582 00000100:00100000:2.0:1789762866.819844:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340056346985 time 340056462329 state 0x2 ref 0 00000100:00100000:1.0:1789762866.821838:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.821892:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067442560 00000100:00100000:3.0:1789762866.821900:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067442560 00000100:00100000:3.0:1789762866.821962:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340056346985 time 340056462329 state 0x0 ref 1 00000100:00100000:3.0:1789762866.821976:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340106462329 time 340056462329 state 0x1 ref 1 00000100:00100000:0.0:1789762866.859467:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340106462329 time 340106540170 state 0x0 ref 1 00000100:00100000:0.0:1789762866.859490:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067442560:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762866.859521:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 94371840, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x2d009 00010000:00010000:0.0:1789762866.859527:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1764fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.859538:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:0.0:1789762866.859541:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762866.859546:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1764fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.859555:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1764fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762866.870115:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967901 is committed 00002000:00010000:3.0:1789762866.870127:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 94371840 -> 95420416 00002000:00010000:3.0:1789762866.870132:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 184329 -> 186377 00010000:00010000:3.0:1789762866.870141:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1764fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.870150:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1764fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.870156:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1764fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.870161:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.870164:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1764fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.870171:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1764fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.870191:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1764fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.870196:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1764fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.870244:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067442560:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10755us (48405us total) trans 4294967901 rc 0/0 00000100:00100000:3.0:1789762866.870250:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 583 00000100:00100000:3.0:1789762866.870258:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340106462329 time 340106540170 state 0x2 ref 0 00000100:00100000:1.0:1789762866.872400:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.872435:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067442688 00000100:00100000:3.0:1789762866.872443:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067442688 00000100:00100000:3.0:1789762866.872477:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340106462329 time 340106540170 state 0x0 ref 1 00000100:00100000:3.0:1789762866.872487:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340156540170 time 340106540170 state 0x1 ref 1 00000100:00100000:0.0:1789762866.909569:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340156540170 time 340156642169 state 0x0 ref 1 00000100:00100000:0.0:1789762866.909584:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067442688:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762866.909617:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 95420416, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x2d809 00010000:00010000:0.0:1789762866.909623:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f176503 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.909634:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:0.0:1789762866.909637:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762866.909642:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f176503 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.909660:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f176503 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762866.920194:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967902 is committed 00002000:00010000:0.0:1789762866.920203:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 95420416 -> 96468992 00002000:00010000:0.0:1789762866.920210:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 186377 -> 188425 00010000:00010000:0.0:1789762866.920224:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f176503 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.920244:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f176503 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.920253:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f176503 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.920261:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762866.920267:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f176503 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.920276:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f176503 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.920297:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f176503 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762866.920305:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f176503 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762866.920422:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067442688:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10836us (48016us total) trans 4294967902 rc 0/0 00000100:00100000:0.0:1789762866.920433:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 584 00000100:00100000:0.0:1789762866.920445:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340156540170 time 340156642169 state 0x2 ref 0 00000100:00100000:2.0:1789762866.923087:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.923128:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067442816 00000100:00100000:3.0:1789762866.923136:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067442816 00000100:00100000:3.0:1789762866.923179:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340156540170 time 340156642169 state 0x0 ref 1 00000100:00100000:3.0:1789762866.923190:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340206642169 time 340156642169 state 0x1 ref 1 00000100:00100000:2.0:1789762866.959636:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340206642169 time 340206707563 state 0x0 ref 1 00000100:00100000:2.0:1789762866.959652:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067442816:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762866.959689:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 96468992, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x2e009 00010000:00010000:2.0:1789762866.959696:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17650a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.959709:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:2.0:1789762866.959712:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762866.959716:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17650a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762866.959726:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17650a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762866.970698:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967903 is committed 00002000:00010000:3.0:1789762866.970710:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 96468992 -> 97517568 00002000:00010000:3.0:1789762866.970720:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 188425 -> 190473 00010000:00010000:3.0:1789762866.970731:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17650a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.970740:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17650a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.970747:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17650a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.970752:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762866.970756:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17650a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.970762:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17650a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.970777:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17650a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762866.970782:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fdda7ea/0xdf5a7ac59f17650a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762866.970833:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067442816:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11184us (47750us total) trans 4294967903 rc 0/0 00000100:00100000:3.0:1789762866.970840:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 585 00000100:00100000:3.0:1789762866.970848:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340206642169 time 340206707563 state 0x2 ref 0 00000100:00100000:0.0:1789762866.972908:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762866.972972:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067442944 00000100:00100000:3.0:1789762866.972980:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067442944 00000100:00100000:3.0:1789762866.973022:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340206642169 time 340206707563 state 0x0 ref 1 00000100:00100000:3.0:1789762866.973032:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340256707563 time 340206707563 state 0x1 ref 1 00000100:00100000:2.0:1789762867.009703:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340256707563 time 340256771608 state 0x0 ref 1 00000100:00100000:2.0:1789762867.009737:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067442944:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:1.0:1789762867.009770:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762867.009790:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 97517568, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x2e809 00010000:00010000:2.0:1789762867.009800:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176511 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762867.009813:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:2.0:1789762867.009817:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762867.009821:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762867.009832:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762867.020499:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967904 is committed 00002000:00010000:3.0:1789762867.020513:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 97517568 -> 98566144 00002000:00010000:3.0:1789762867.020517:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 190473 -> 192521 00010000:00010000:3.0:1789762867.020528:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.020541:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176511 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.020551:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176511 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.020557:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762867.020561:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176511 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.020567:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176511 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.020580:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176511 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.020596:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176511 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762867.020649:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067442944:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10930us (47739us total) trans 4294967904 rc 0/0 00000100:00100000:3.0:1789762867.020656:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 586 00000100:00100000:3.0:1789762867.020664:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340256707563 time 340256771608 state 0x2 ref 0 00000100:00100000:1.0:1789762867.022868:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.022946:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067443072 00000100:00100000:0.0:1789762867.022956:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067443072 00000100:00100000:0.0:1789762867.022995:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340256707563 time 340256771608 state 0x0 ref 1 00000100:00100000:0.0:1789762867.023005:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340306771608 time 340256771608 state 0x1 ref 1 00000100:00100000:1.0:1789762867.059778:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340306771608 time 340306849638 state 0x0 ref 1 00000100:00100000:1.0:1789762867.059793:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067443072:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762867.059829:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 98566144, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x2f009 00010000:00010000:1.0:1789762867.059838:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176518 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.059854:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:1.0:1789762867.059858:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762867.059865:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176518 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.059878:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176518 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762867.072158:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 98566144 -> 99614720 00000001:00080000:2.0:1789762867.072162:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967905 is committed 00002000:00010000:1.0:1789762867.072164:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 192521 -> 194569 00010000:00010000:1.0:1789762867.072174:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176518 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.072184:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176518 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.072190:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176518 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.072196:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762867.072199:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176518 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.072205:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176518 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.072217:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176518 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.072222:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f176518 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762867.072270:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067443072:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12480us (49404us total) trans 4294967905 rc 0/0 00000100:00100000:1.0:1789762867.072277:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 587 00000100:00100000:1.0:1789762867.072284:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340306771608 time 340306849638 state 0x2 ref 0 00000100:00100000:0.0:1789762867.074389:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762867.074424:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067443200 00000100:00100000:3.0:1789762867.074431:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067443200 00000100:00100000:3.0:1789762867.074464:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340306771608 time 340306849638 state 0x0 ref 1 00000100:00100000:3.0:1789762867.074473:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340356849638 time 340306849638 state 0x1 ref 1 00000100:00100000:0.0:1789762867.109936:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340356849638 time 340356991236 state 0x0 ref 1 00000100:00100000:0.0:1789762867.109955:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79ac050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067443200:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762867.109992:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 99614720, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x2f809 00010000:00010000:0.0:1789762867.109998:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17651f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.110011:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:0.0:1789762867.110015:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.110030:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17651f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.110041:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17651f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762867.121104:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967906 is committed 00002000:00010000:0.0:1789762867.121122:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 99614720 -> 100663296 00002000:00010000:0.0:1789762867.121127:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 194569 -> 196617 00010000:00010000:0.0:1789762867.121136:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17651f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.121145:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17651f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.121152:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17651f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.121157:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762867.121161:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17651f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.121168:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17651f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.121181:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17651f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.121196:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17651f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762867.121251:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79ac050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067443200:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11301us (46861us total) trans 4294967906 rc 0/0 00000100:00100000:0.0:1789762867.121258:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 588 00000100:00100000:0.0:1789762867.121267:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340356849638 time 340356991236 state 0x2 ref 0 00000100:00100000:1.0:1789762867.123397:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762867.123436:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067443328 00000100:00100000:3.0:1789762867.123444:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067443328 00000100:00100000:3.0:1789762867.123480:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340356849638 time 340356991236 state 0x0 ref 1 00000100:00100000:3.0:1789762867.123491:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340406991236 time 340356991236 state 0x1 ref 1 00000100:00100000:0.0:1789762867.159992:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340406991236 time 340407064120 state 0x0 ref 1 00000100:00100000:0.0:1789762867.160007:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067443328:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762867.160040:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 100663296, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x30009 00010000:00010000:0.0:1789762867.160045:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176526 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.160057:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:0.0:1789762867.160060:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.160065:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.160075:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762867.170597:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967907 is committed 00002000:00010000:0.0:1789762867.170609:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 100663296 -> 101711872 00002000:00010000:0.0:1789762867.170619:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 196617 -> 198667 00010000:00010000:0.0:1789762867.170628:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.170638:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176526 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.170644:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176526 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.170649:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762867.170652:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176526 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.170659:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176526 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.170673:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176526 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.170678:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176526 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762867.170727:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067443328:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10722us (47329us total) trans 4294967907 rc 0/0 00000100:00100000:0.0:1789762867.170735:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 589 00000100:00100000:0.0:1789762867.170742:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340406991236 time 340407064120 state 0x2 ref 0 00000100:00100000:2.0:1789762867.172728:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762867.172801:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067443456 00000100:00100000:1.0:1789762867.172808:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067443456 00000100:00100000:1.0:1789762867.172844:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340406991236 time 340407064120 state 0x0 ref 1 00000100:00100000:1.0:1789762867.172854:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340457064120 time 340407064120 state 0x1 ref 1 00000100:00100000:0.0:1789762867.210056:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340457064120 time 340457123871 state 0x0 ref 1 00000100:00100000:0.0:1789762867.210075:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca559180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067443456:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762867.210117:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 101711872, mtime: 0x6aad9d32, atime: 0x0, ctime: 0x6aad9d32, blocks: 0x3080b 00010000:00010000:0.0:1789762867.210124:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17652d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.210137:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:0.0:1789762867.210141:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.210145:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17652d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.210156:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17652d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762867.220893:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967908 is committed 00002000:00010000:0.0:1789762867.220950:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 101711872 -> 102760448 00002000:00010000:0.0:1789762867.220955:0:8058:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb mtime from disk: 1789762866 -> 1789762867 00002000:00010000:0.0:1789762867.220957:0:8058:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb ctime from disk: 1789762866 -> 1789762867 00002000:00010000:0.0:1789762867.220959:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 198667 -> 200715 00010000:00010000:0.0:1789762867.220966:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17652d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.220975:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17652d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.220993:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17652d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.220998:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762867.221001:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17652d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.221007:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17652d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.221020:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17652d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.221026:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f17652d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762867.221078:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca559180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067443456:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11009us (48351us total) trans 4294967908 rc 0/0 00000100:00100000:0.0:1789762867.221087:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 590 00000100:00100000:0.0:1789762867.221095:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340457064120 time 340457123871 state 0x2 ref 0 00000100:00100000:1.0:1789762867.223211:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.223277:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067443584 00000100:00100000:0.0:1789762867.223286:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067443584 00000100:00100000:0.0:1789762867.223319:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340457064120 time 340457123871 state 0x0 ref 1 00000100:00100000:0.0:1789762867.223352:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340507123871 time 340457123871 state 0x1 ref 1 00000100:00100000:2.0:1789762867.260165:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340507123871 time 340507233944 state 0x0 ref 1 00000100:00100000:2.0:1789762867.260185:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067443584:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762867.260226:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 102760448, mtime: 0x6aad9d33, atime: 0x0, ctime: 0x6aad9d33, blocks: 0x3100b 00010000:00010000:2.0:1789762867.260243:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176534 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762867.260255:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:2.0:1789762867.260259:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762867.260263:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176534 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762867.260274:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176534 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762867.271571:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967909 is committed 00002000:00010000:0.0:1789762867.271579:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 102760448 -> 103809024 00002000:00010000:0.0:1789762867.271585:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 200715 -> 202763 00010000:00010000:0.0:1789762867.271594:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176534 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.271603:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176534 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.271610:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176534 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.271615:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762867.271618:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176534 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.271625:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176534 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.271638:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176534 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.271643:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d095fc79/0xdf5a7ac59f176534 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762867.271710:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067443584:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11530us (48498us total) trans 4294967909 rc 0/0 00000100:00100000:0.0:1789762867.271717:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 591 00000100:00100000:0.0:1789762867.271726:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340507123871 time 340507233944 state 0x2 ref 0 00000100:00100000:1.0:1789762867.273808:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.273848:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067443712 00000100:00100000:0.0:1789762867.273857:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067443712 00000100:00100000:0.0:1789762867.273891:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340507123871 time 340507233944 state 0x0 ref 1 00000100:00100000:0.0:1789762867.273902:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340557233944 time 340507233944 state 0x1 ref 1 00000100:00100000:3.0:1789762867.310262:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340557233944 time 340557334714 state 0x0 ref 1 00000100:00100000:3.0:1789762867.310293:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067443712:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762867.310332:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 103809024, mtime: 0x6aad9d33, atime: 0x0, ctime: 0x6aad9d33, blocks: 0x3180b 00010000:00010000:3.0:1789762867.310353:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f17653b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.310366:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:3.0:1789762867.310369:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.310373:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f17653b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.310385:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f17653b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762867.322070:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967910 is committed 00002000:00010000:0.0:1789762867.322122:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb size from disk: 103809024 -> 104857600 00002000:00010000:0.0:1789762867.322127:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x7:0x0] updating lvb blocks from disk: 202763 -> 204811 00010000:00010000:0.0:1789762867.322134:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f17653b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.322144:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f17653b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.322150:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f17653b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.322155:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762867.322158:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f17653b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.322171:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f17653b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.322184:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f17653b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.322189:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f17653b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762867.322242:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067443712:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11952us (48433us total) trans 4294967910 rc 0/0 00000100:00100000:0.0:1789762867.322249:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 592 00000100:00100000:0.0:1789762867.322258:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340557233944 time 340557334714 state 0x2 ref 0 00000100:00100000:2.0:1789762867.323762:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.323809:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067443968 00000100:00100000:0.0:1789762867.323816:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067443968 00000100:00100000:0.0:1789762867.323860:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 79 00000100:00100000:0.0:1789762867.323869:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca558a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:14597:x1876702067443968:12345-192.168.203.37@tcp:35:dd.500 00000100:00080000:1.0:1789762867.323904:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789762867.324059:0:6421:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967413 00000100:00100000:0.0:1789762867.324120:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca558a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:14597:x1876702067443968:12345-192.168.203.37@tcp:35:dd.500 Request processed in 253us (359us total) trans 4294967414 rc 0/0 00000100:00100000:0.0:1789762867.324126:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 79 00000020:00080000:0.0:1789762867.324130:0:6421:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1789762867.365641:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762867.365683:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067444224 00000100:00100000:3.0:1789762867.365691:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067444224 00000100:00100000:3.0:1789762867.365740:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 286 00000100:00100000:3.0:1789762867.365749:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:14613:x1876702067444224:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762867.365768:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762867.365782:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f176542 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8ef expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.365868:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f176549 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.365879:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f176549 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.365901:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f176549 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762867.365911:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x18:0x0]: returning size 104857600 00010000:00010000:3.0:1789762867.366016:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273ce00/0xdf5a7ac59f176542 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8ef expref: 23 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.366028:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f176549 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8ef expref: 22 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.366035:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff9c37e125a200) 00010000:00010000:3.0:1789762867.366038:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:3.0:1789762867.366040:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d273d200, rc 0) 00000100:00100000:3.0:1789762867.366087:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:14613:x1876702067444224:12345-192.168.203.37@tcp:101:dd.500 Request processed in 340us (451us total) trans 4294967415 rc 0/0 00000100:00100000:3.0:1789762867.366092:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 286 00000100:00100000:1.0:1789762867.367373:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.367415:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067444352 00000100:00100000:0.0:1789762867.367420:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067444352 00000100:00100000:0.0:1789762867.367457:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 171 00000100:00100000:0.0:1789762867.367463:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067444352:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:0.0:1789762867.367470:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762867.367481:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176557 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8f6 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762867.367496:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 104857600, mtime: 0x6aad9d33, atime: 0x0, ctime: 0x6aad9d33, blocks: 0x3200b 00010000:00010000:0.0:1789762867.367507:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=340s v=(0 0) 00010000:00010000:0.0:1789762867.367511:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.367517:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176557 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8f6 expref: 6 pid: 8054 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.367527:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176557 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8f6 expref: 6 pid: 8054 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.367534:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769e00, rc 0) 00000100:00100000:0.0:1789762867.367561:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067444352:12345-192.168.203.37@tcp:101:dd.500 Request processed in 98us (189us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762867.367566:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 171 00000100:00100000:2.0:1789762867.369682:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.369722:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067444480 00000100:00100000:0.0:1789762867.369728:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067444480 00000100:00100000:0.0:1789762867.369758:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278898693604 time 278898753179 state 0x0 ref 1 00000100:00100000:0.0:1789762867.369766:0:8058:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 6: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34b00 gen 0 rate 10 depth 3 token 0 deadline 278898693604 time 278898753179 state 0x0 ref 1 00000100:00100000:0.0:1789762867.369785:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 3 deadline 340666861706 time 340616861706 state 0x1 ref 1 00000100:00100000:0.0:1789762867.369815:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 2 deadline 340666861706 time 340616892913 state 0x0 ref 1 00000100:00100000:0.0:1789762867.369824:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067444480:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762867.369841:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17655e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.369849:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.369852:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.369855:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17655e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.369862:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17655e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.373107:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17655e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.373118:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17655e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.373124:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17655e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.373129:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762867.373132:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17655e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.373148:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17655e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.373159:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17655e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.373164:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17655e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762867.373202:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067444480:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3379us (3521us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762867.373209:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 593 00000100:00100000:0.0:1789762867.373216:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 2 deadline 340666861706 time 340616892913 state 0x2 ref 0 00000100:00100000:1.0:1789762867.374652:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762867.374684:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067444608 00000100:00100000:3.0:1789762867.374689:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067444608 00000100:00100000:3.0:1789762867.374711:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 2 deadline 340666861706 time 340616892913 state 0x0 ref 1 00000100:00100000:3.0:1789762867.374720:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 2 deadline 340666892913 time 340616892913 state 0x1 ref 1 00000100:00100000:3.0:1789762867.374737:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 1 deadline 340666892913 time 340621816648 state 0x0 ref 1 00000100:00100000:3.0:1789762867.374746:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067444608:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762867.374769:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176565 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.374778:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.374780:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.374784:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176565 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.374791:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176565 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.377037:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176565 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.377047:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176565 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.377054:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176565 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.377061:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762867.377065:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176565 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.377074:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176565 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.377103:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176565 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.377110:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fbcf3970/0xdf5a7ac59f176565 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762867.377148:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067444608:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2401us (2496us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762867.377154:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 594 00000100:00100000:3.0:1789762867.377161:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 1 deadline 340666892913 time 340621816648 state 0x2 ref 0 00000100:00100000:1.0:1789762867.378580:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762867.378612:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067444736 00000100:00100000:3.0:1789762867.378617:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067444736 00000100:00100000:3.0:1789762867.378639:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 1 deadline 340666892913 time 340621816648 state 0x0 ref 1 00000100:00100000:3.0:1789762867.378647:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 1 deadline 340671816648 time 340621816648 state 0x1 ref 1 00000100:00100000:3.0:1789762867.378665:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340671816648 time 340625744031 state 0x0 ref 1 00000100:00100000:3.0:1789762867.378673:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067444736:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762867.378685:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17656c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.378693:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.378700:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.378704:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17656c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.378711:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17656c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.381005:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17656c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.381014:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17656c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.381020:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17656c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.381025:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762867.381028:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17656c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.381033:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17656c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.381043:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17656c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.381048:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17656c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762867.381103:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067444736:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2429us (2522us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762867.381113:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 595 00000100:00100000:3.0:1789762867.381121:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340671816648 time 340625744031 state 0x2 ref 0 00000100:00100000:1.0:1789762867.382638:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.382711:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067444864 00000100:00100000:0.0:1789762867.382716:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067444864 00000100:00100000:0.0:1789762867.382740:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340671816648 time 340625744031 state 0x0 ref 1 00000100:00100000:0.0:1789762867.382749:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340675744031 time 340625744031 state 0x1 ref 1 00000100:00100000:0.0:1789762867.428718:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340675744031 time 340675789470 state 0x0 ref 1 00000100:00100000:0.0:1789762867.428734:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067444864:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762867.428757:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176573 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.428776:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.428779:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.428784:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176573 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.428792:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176573 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.431479:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176573 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.431489:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176573 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.431495:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176573 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.431500:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762867.431503:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176573 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.431509:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176573 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.431521:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176573 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.431537:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176573 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762867.431588:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067444864:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2857us (48950us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762867.431594:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 596 00000100:00100000:0.0:1789762867.431602:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340675744031 time 340675789470 state 0x2 ref 0 00000100:00100000:2.0:1789762867.433334:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.433382:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067444992 00000100:00100000:0.0:1789762867.433389:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067444992 00000100:00100000:0.0:1789762867.433418:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340675744031 time 340675789470 state 0x0 ref 1 00000100:00100000:0.0:1789762867.433429:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340725789470 time 340675789470 state 0x1 ref 1 00000100:00100000:0.0:1789762867.478769:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340725789470 time 340725835011 state 0x0 ref 1 00000100:00100000:0.0:1789762867.478788:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca558380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067444992:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762867.478821:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17657a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.478833:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.478837:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.478841:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17657a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.478867:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17657a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.481366:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17657a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.481377:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17657a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.481384:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17657a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.481390:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762867.481394:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17657a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.481401:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17657a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.481415:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17657a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.481420:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17657a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762867.481469:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca558380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067444992:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2686us (48134us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762867.481484:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 597 00000100:00100000:0.0:1789762867.481492:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340725789470 time 340725835011 state 0x2 ref 0 00000100:00100000:2.0:1789762867.483530:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762867.483571:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067445120 00000100:00100000:3.0:1789762867.483580:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067445120 00000100:00100000:3.0:1789762867.483615:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340725789470 time 340725835011 state 0x0 ref 1 00000100:00100000:3.0:1789762867.483626:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340775835011 time 340725835011 state 0x1 ref 1 00000100:00100000:3.0:1789762867.528808:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340775835011 time 340775879860 state 0x0 ref 1 00000100:00100000:3.0:1789762867.528823:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067445120:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762867.528847:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176581 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.528867:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.528870:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.528874:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176581 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.528883:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176581 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.531672:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176581 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.531683:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176581 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.531689:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176581 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.531694:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762867.531697:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176581 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.531702:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176581 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.531713:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176581 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.531718:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176581 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762867.531771:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067445120:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2950us (48241us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762867.531781:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 598 00000100:00100000:3.0:1789762867.531797:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340775835011 time 340775879860 state 0x2 ref 0 00000100:00100000:1.0:1789762867.533623:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.533666:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067445248 00000100:00100000:0.0:1789762867.533673:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067445248 00000100:00100000:0.0:1789762867.533703:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340775835011 time 340775879860 state 0x0 ref 1 00000100:00100000:0.0:1789762867.533713:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340825879860 time 340775879860 state 0x1 ref 1 00000100:00100000:3.0:1789762867.578949:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340825879860 time 340825993951 state 0x0 ref 1 00000100:00100000:3.0:1789762867.578970:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067445248:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762867.579004:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176588 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.579022:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.579027:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.579034:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176588 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.579049:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176588 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.581613:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176588 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.581622:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176588 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.581629:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176588 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.581634:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762867.581648:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176588 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.581655:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176588 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.581670:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176588 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.581676:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176588 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762867.581733:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067445248:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2763us (48103us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762867.581741:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 599 00000100:00100000:3.0:1789762867.581750:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340825879860 time 340825993951 state 0x2 ref 0 00000100:00100000:3.0:1789762867.583741:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.583792:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067445376 00000100:00100000:0.0:1789762867.583800:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067445376 00000100:00100000:0.0:1789762867.583838:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340825879860 time 340825993951 state 0x0 ref 1 00000100:00100000:0.0:1789762867.583848:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340875993951 time 340825993951 state 0x1 ref 1 00000100:00100000:0.0:1789762867.628969:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340875993951 time 340876040209 state 0x0 ref 1 00000100:00100000:0.0:1789762867.628985:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9fffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067445376:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762867.629010:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17658f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.629022:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.629026:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.629030:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17658f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.629047:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17658f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.631461:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17658f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.631471:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17658f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.631478:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17658f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.631483:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762867.631486:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17658f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.631492:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17658f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.631502:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17658f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.631507:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17658f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762867.631552:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9fffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067445376:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2570us (47813us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762867.631564:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 600 00000100:00100000:0.0:1789762867.631572:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340875993951 time 340876040209 state 0x2 ref 0 00000100:00100000:2.0:1789762867.633202:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762867.633240:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067445504 00000100:00100000:3.0:1789762867.633246:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067445504 00000100:00100000:3.0:1789762867.633278:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340875993951 time 340876040209 state 0x0 ref 1 00000100:00100000:3.0:1789762867.633287:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340926040209 time 340876040209 state 0x1 ref 1 00000100:00100000:3.0:1789762867.679014:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340926040209 time 340926082235 state 0x0 ref 1 00000100:00100000:3.0:1789762867.679049:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca558e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067445504:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762867.679074:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176596 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.679086:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.679089:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.679093:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176596 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.679101:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176596 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.681687:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176596 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.681697:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176596 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.681703:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176596 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.681708:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762867.681712:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176596 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.681717:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176596 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.681737:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176596 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.681743:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f176596 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762867.681791:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca558e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067445504:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2748us (48590us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762867.681798:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 601 00000100:00100000:3.0:1789762867.681806:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340926040209 time 340926082235 state 0x2 ref 0 00000100:00100000:2.0:1789762867.683471:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762867.683503:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067445632 00000100:00100000:3.0:1789762867.683510:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067445632 00000100:00100000:3.0:1789762867.683539:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340926040209 time 340926082235 state 0x0 ref 1 00000100:00100000:3.0:1789762867.683549:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340976082235 time 340926082235 state 0x1 ref 1 00000100:00100000:2.0:1789762867.712046:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074700544:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762867.712105:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762867.712146:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074700544 00000100:00100000:3.0:1789762867.712153:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074700544 00000100:00100000:3.0:1789762867.712198:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:3.0:1789762867.712208:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca559880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074700544:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789762867.712237:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789762867.712286:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca559880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074700544:12345-0@lo:41:osp-pre-0-1.0 Request processed in 81us (184us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762867.712293:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000100:00100000:2.0:1789762867.712329:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074700544:0@lo:41:osp-pre-0-1.0 00000001:00080000:3.0:1789762867.719796:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967414 is committed 00010000:00010000:1.0:1789762867.719812:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719823:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762867.719823:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967415 is committed 00010000:00010000:1.0:1789762867.719832:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719838:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762867.719842:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719847:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719858:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743300) 00010000:00010000:1.0:1789762867.719860:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762867.719863:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719871:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7c00/0xdf5a7ac59f1761ad lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719885:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1761b4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719889:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1761b4 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719893:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1761b4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719897:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762867.719898:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1761b4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719902:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1761b4 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719906:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:1.0:1789762867.719908:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762867.719910:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1761b4 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719913:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6000/0xdf5a7ac59f1761b4 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719945:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f17620f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719950:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f17620f lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719955:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f17620f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719958:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc7800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762867.719960:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f17620f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719964:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f17620f lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719969:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:1.0:1789762867.719970:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789762867.719972:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f17620f lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719976:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc7800/0xdf5a7ac59f17620f lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719981:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176232 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719985:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176232 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719990:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176232 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.719993:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bc6400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762867.719994:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176232 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.720001:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176232 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.720006:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:1.0:1789762867.720007:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789762867.720009:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176232 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.720012:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c9bc6400/0xdf5a7ac59f176232 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.720018:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f176263 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.720021:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f176263 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.720025:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f176263 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.720028:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762867.720030:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f176263 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.720033:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f176263 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.720037:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x4009f790 (ffff9c37e125ab00) 00010000:00010000:1.0:1789762867.720039:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762867.720040:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f176263 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762867.720045:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f176263 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762867.729069:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340976082235 time 340976137980 state 0x0 ref 1 00000100:00100000:3.0:1789762867.729083:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca559500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067445632:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762867.729106:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17659d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.729118:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.729122:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.729126:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17659d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.729135:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17659d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.731586:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17659d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.731599:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17659d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.731607:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17659d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.731615:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762867.731619:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17659d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.731637:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17659d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.731654:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17659d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.731661:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f17659d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762867.731703:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca559500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067445632:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2623us (48233us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762867.731709:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 602 00000100:00100000:3.0:1789762867.731716:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340976082235 time 340976137980 state 0x2 ref 0 00000100:00100000:2.0:1789762867.733562:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.733613:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067445760 00000100:00100000:0.0:1789762867.733620:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067445760 00000100:00100000:0.0:1789762867.733653:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340976082235 time 340976137980 state 0x0 ref 1 00000100:00100000:0.0:1789762867.733663:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341026137980 time 340976137980 state 0x1 ref 1 00000100:00100000:0.0:1789762867.779124:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341026137980 time 341026196075 state 0x0 ref 1 00000100:00100000:0.0:1789762867.779140:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca559c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067445760:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762867.779164:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765a4 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.779176:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762867.779179:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.779184:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765a4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.779195:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765a4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.781792:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765a4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.781814:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765a4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.781822:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765a4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.781827:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762867.781831:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765a4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.781837:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765a4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.781850:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765a4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.781855:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765a4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762867.781897:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca559c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067445760:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2761us (48335us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762867.781904:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 603 00000100:00100000:0.0:1789762867.781910:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341026137980 time 341026196075 state 0x2 ref 0 00000100:00100000:1.0:1789762867.783600:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762867.783643:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067445888 00000100:00100000:3.0:1789762867.783651:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067445888 00000100:00100000:3.0:1789762867.783689:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341026137980 time 341026196075 state 0x0 ref 1 00000100:00100000:3.0:1789762867.783700:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341076196075 time 341026196075 state 0x1 ref 1 00000100:00100000:0.0:1789762867.829217:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341076196075 time 341076288487 state 0x0 ref 1 00000100:00100000:0.0:1789762867.829242:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067445888:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762867.829269:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765ab lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.829281:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.829284:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.829289:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765ab lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.829297:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765ab lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.831634:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765ab lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.831645:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765ab lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.831651:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765ab lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.831655:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762867.831658:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765ab lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.831664:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765ab lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.831682:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765ab lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.831688:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f1765ab lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762867.831733:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067445888:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2494us (48134us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762867.831740:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 604 00000100:00100000:0.0:1789762867.831747:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341076196075 time 341076288487 state 0x2 ref 0 00000100:00100000:2.0:1789762867.833481:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762867.833522:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067446016 00000100:00100000:3.0:1789762867.833529:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067446016 00000100:00100000:3.0:1789762867.833561:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341076196075 time 341076288487 state 0x0 ref 1 00000100:00100000:3.0:1789762867.833570:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341126288487 time 341076288487 state 0x1 ref 1 00010000:00010000:1.0:1789762867.840020:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762867.879277:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341126288487 time 341126345655 state 0x0 ref 1 00000100:00100000:3.0:1789762867.879296:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067446016:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762867.879332:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765b2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.879361:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.879364:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762867.879369:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765b2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.879381:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765b2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.881846:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765b2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.881857:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765b2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.881867:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765b2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.881876:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762867.881882:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765b2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.881903:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765b2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.881939:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765b2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.881946:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765b2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762867.881991:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067446016:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2701us (48510us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762867.882001:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 605 00000100:00100000:3.0:1789762867.882010:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341126288487 time 341126345655 state 0x2 ref 0 00000100:00100000:2.0:1789762867.883795:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.883874:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067446144 00000100:00100000:0.0:1789762867.883881:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067446144 00000100:00100000:0.0:1789762867.883941:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341126288487 time 341126345655 state 0x0 ref 1 00000100:00100000:0.0:1789762867.883952:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341176345655 time 341126345655 state 0x1 ref 1 00000100:00100000:2.0:1789762867.929398:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341176345655 time 341176465219 state 0x0 ref 1 00000100:00100000:2.0:1789762867.929426:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca558700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067446144:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762867.929450:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1765b9 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762867.929462:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762867.929465:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762867.929470:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1765b9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762867.929479:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1765b9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.932392:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1765b9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.932402:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1765b9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.932412:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1765b9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.932417:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762867.932420:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1765b9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.932433:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1765b9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.932445:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1765b9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.932450:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1765b9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762867.932494:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca558700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067446144:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3071us (48701us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762867.932501:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 606 00000100:00100000:3.0:1789762867.932508:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341176345655 time 341176465219 state 0x2 ref 0 00000100:00100000:1.0:1789762867.934361:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762867.934422:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067446272 00000100:00100000:0.0:1789762867.934430:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067446272 00000100:00100000:0.0:1789762867.934462:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341176345655 time 341176465219 state 0x0 ref 1 00000100:00100000:0.0:1789762867.934472:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341226465219 time 341176465219 state 0x1 ref 1 00000100:00100000:0.0:1789762867.979452:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341226465219 time 341226518081 state 0x0 ref 1 00000100:00100000:0.0:1789762867.979483:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d275aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067446272:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762867.979518:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ad056a1f/0xdf5a7ac59f1765c0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.979532:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.979535:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762867.979540:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad056a1f/0xdf5a7ac59f1765c0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762867.979550:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad056a1f/0xdf5a7ac59f1765c0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.983083:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ad056a1f/0xdf5a7ac59f1765c0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.983110:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad056a1f/0xdf5a7ac59f1765c0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.983124:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad056a1f/0xdf5a7ac59f1765c0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.983136:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee800000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762867.983144:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad056a1f/0xdf5a7ac59f1765c0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.983173:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad056a1f/0xdf5a7ac59f1765c0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.983200:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad056a1f/0xdf5a7ac59f1765c0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762867.983210:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad056a1f/0xdf5a7ac59f1765c0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762867.983295:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d275aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067446272:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3812us (48931us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762867.983307:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 607 00000100:00100000:3.0:1789762867.983321:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341226465219 time 341226518081 state 0x2 ref 0 00000100:00100000:1.0:1789762867.985912:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762867.986030:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067446400 00000100:00100000:3.0:1789762867.986046:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067446400 00000100:00100000:3.0:1789762867.986116:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341226465219 time 341226518081 state 0x0 ref 1 00000100:00100000:3.0:1789762867.986135:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341276518081 time 341226518081 state 0x1 ref 1 00000100:00100000:3.0:1789762868.029565:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341276518081 time 341276626761 state 0x0 ref 1 00000100:00100000:3.0:1789762868.029594:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff617480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067446400:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762868.029634:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765c7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.029664:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.029669:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.029677:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765c7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.029692:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765c7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.032995:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765c7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.033023:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765c7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.033042:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765c7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.033057:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762868.033066:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765c7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.033086:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765c7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.033116:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765c7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.033147:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765c7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762868.033231:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff617480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067446400:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3643us (47322us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762868.033244:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 608 00000100:00100000:3.0:1789762868.033258:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341276518081 time 341276626761 state 0x2 ref 0 00000100:00100000:3.0:1789762868.035664:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762868.035722:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067446528 00000100:00100000:2.0:1789762868.035731:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067446528 00000100:00100000:2.0:1789762868.035775:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341276518081 time 341276626761 state 0x0 ref 1 00000100:00100000:2.0:1789762868.035788:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341326626761 time 341276626761 state 0x1 ref 1 00000100:00100000:3.0:1789762868.079668:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341326626761 time 341326733510 state 0x0 ref 1 00000100:00100000:3.0:1789762868.079693:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9fff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067446528:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762868.079749:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765ce lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.079776:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.079785:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.079796:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765ce lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.079822:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765ce lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.084447:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765ce lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.084481:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765ce lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.084495:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765ce lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.084506:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762868.084512:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765ce lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.084537:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765ce lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.084567:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765ce lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.084580:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765ce lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762868.084680:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9fff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067446528:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4989us (49010us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762868.084698:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 609 00000100:00100000:3.0:1789762868.084713:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341326626761 time 341326733510 state 0x2 ref 0 00000100:00100000:3.0:1789762868.088426:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.088538:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067446656 00000100:00100000:0.0:1789762868.088552:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067446656 00000100:00100000:0.0:1789762868.088611:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341326626761 time 341326733510 state 0x0 ref 1 00000100:00100000:0.0:1789762868.088643:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341376733510 time 341326733510 state 0x1 ref 1 00000100:00100000:0.0:1789762868.129759:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341376733510 time 341376820491 state 0x0 ref 1 00000100:00100000:0.0:1789762868.129806:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067446656:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762868.129863:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765d5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.129893:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.129912:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.129971:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765d5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.129992:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765d5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.134893:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765d5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.134914:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765d5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.134975:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765d5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.134987:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d10ba000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762868.134993:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765d5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.135004:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765d5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.135026:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765d5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.135036:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765d5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762868.135167:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067446656:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5382us (46740us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762868.135181:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 610 00000100:00100000:0.0:1789762868.135194:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341376733510 time 341376820491 state 0x2 ref 0 00000100:00100000:3.0:1789762868.138158:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.138225:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067446784 00000100:00100000:0.0:1789762868.138238:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067446784 00000100:00100000:0.0:1789762868.138299:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341376733510 time 341376820491 state 0x0 ref 1 00000100:00100000:0.0:1789762868.138325:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341426820491 time 341376820491 state 0x1 ref 1 00000100:00100000:0.0:1789762868.179826:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341426820491 time 341426889836 state 0x0 ref 1 00000100:00100000:0.0:1789762868.179856:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067446784:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762868.179894:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765dc lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.179912:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.179944:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.179952:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765dc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.179967:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765dc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.184013:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765dc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.184030:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765dc lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.184041:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765dc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.184050:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d10ba000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762868.184056:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765dc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.184069:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765dc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.184094:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765dc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.184109:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765dc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762868.184186:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067446784:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4338us (46030us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762868.184199:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 611 00000100:00100000:0.0:1789762868.184220:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341426820491 time 341426889836 state 0x2 ref 0 00000100:00100000:3.0:1789762868.186496:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762868.186561:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067446912 00000100:00100000:1.0:1789762868.186571:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067446912 00000100:00100000:1.0:1789762868.186614:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341426820491 time 341426889836 state 0x0 ref 1 00000100:00100000:1.0:1789762868.186628:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341476889836 time 341426889836 state 0x1 ref 1 00000100:00100000:3.0:1789762868.229889:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341476889836 time 341476958059 state 0x0 ref 1 00000100:00100000:3.0:1789762868.229911:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067446912:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762868.229969:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765e3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.229982:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.229986:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.229991:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765e3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.230002:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765e3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789762868.231153:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 16 committed 00000004:00080000:1.0:1789762868.231172:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 16 committed 00000020:00080000:1.0:1789762868.231186:0:7221:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 10 batchid updated 00000020:00080000:1.0:1789762868.231193:0:7221:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 9 batchid updated 00000020:00080000:0.0:1789762868.231219:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 10 00000100:00100000:1.0:1789762868.231249:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca55b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074700672:0@lo:6:osp-syn-0-1.0 00000004:00080000:0.0:1789762868.231269:0:7233:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c36c8976000 with version 22 00000020:00080000:0.0:1789762868.231284:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 9 cancel update log [0x200000401:0x2:0x0].9: rc = 0 00000100:00100000:1.0:1789762868.231300:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1789762868.231303:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 22 rpc_version 22 00000020:00100000:2.0:1789762868.231326:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1789762868.231347:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c36ca559f80 x1876702074700928/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762868.231364:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca559f80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074700928:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1789762868.231383:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074700672 00000100:00100000:1.0:1789762868.231389:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074700672 00000100:00100000:2.0:1789762868.231404:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762868.231435:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252f840 going to sleep for 11 seconds 00000100:00100000:1.0:1789762868.231441:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:3.0:1789762868.231446:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789762868.231456:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074700672:12345-0@lo:6:osp-syn-0-1.0 00000100:00100000:3.0:1789762868.231462:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074701056:0@lo:6:osp-syn-1-1.0 00002000:00080000:1.0:1789762868.231482:0:8052:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:11 count 1 00000100:00100000:3.0:1789762868.231501:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00010000:1.0:1789762868.231510:0:8052:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xb:0x0] initial LVB size: 8388608, mtime: 0x6aad9d2d, atime: 0x0, ctime: 0x6aad9d2d, blocks: 0x4007 00010000:00010000:1.0:1789762868.231517:0:8052:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f1765ea lrc: 3/0,1 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.231527:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=341s v=(0 0) 00010000:00010000:1.0:1789762868.231530:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762868.231533:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f1765ea lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.231541:0:8052:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f1765ea lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.231548:0:8052:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f1765ea lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.231553:0:8052:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f1765ea lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.231566:0:8052:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1789762868.231796:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074701056 00000100:00100000:2.0:1789762868.231800:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074701056 00000100:00100000:2.0:1789762868.231835:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:2.0:1789762868.231843:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074701056:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:2.0:1789762868.231857:0:9038:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:11 count 1 00002000:00010000:2.0:1789762868.231877:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0xb:0x0] initial LVB size: 8388608, mtime: 0x6aad9d2d, atime: 0x0, ctime: 0x6aad9d2d, blocks: 0x4007 00010000:00010000:2.0:1789762868.231883:0:9038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000974c0faa/0xdf5a7ac59f1765f1 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1789762868.231886:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 9 cancel update log [0x240000400:0x2:0x0].9: rc = 0 00010000:00010000:2.0:1789762868.231899:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=341s v=(0 0) 00010000:00010000:2.0:1789762868.231902:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762868.231905:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000974c0faa/0xdf5a7ac59f1765f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762868.231940:0:7233:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c36c89763c0 with version 23 00010000:00010000:2.0:1789762868.231944:0:9038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000974c0faa/0xdf5a7ac59f1765f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1789762868.231945:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 10 cancel update log [0x200000401:0x2:0x0].10: rc = 0 00010000:00010000:2.0:1789762868.231957:0:9038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000974c0faa/0xdf5a7ac59f1765f1 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762868.231963:0:9038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000974c0faa/0xdf5a7ac59f1765f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762868.231976:0:9038:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1789762868.232000:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1789762868.232084:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f1765ea lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.232094:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762868.232098:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f1765ea lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.232116:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f1765ea lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.232131:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f1765ea lrc: 1/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.232137:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a09193fe/0xdf5a7ac59f1765ea lrc: 0/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.232159:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762868.232163:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000974c0faa/0xdf5a7ac59f1765f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.232168:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762868.232171:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000974c0faa/0xdf5a7ac59f1765f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.232178:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000974c0faa/0xdf5a7ac59f1765f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.232188:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000974c0faa/0xdf5a7ac59f1765f1 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.232192:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000974c0faa/0xdf5a7ac59f1765f1 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.232220:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762868.232222:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762868.232225:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:0.0:1789762868.232261:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 10 cancel update log [0x240000400:0x2:0x0].10: rc = 0 00000020:00080000:0.0:1789762868.232272:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 10 00000100:00100000:1.0:1789762868.232419:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074700672:12345-0@lo:6:osp-syn-0-1.0 Request processed in 969us (1118us total) trans 4294967911 rc 0/0 00000100:00100000:1.0:1789762868.232427:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00080000:1.0:1789762868.232458:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000004:00080000:0.0:1789762868.232459:0:3427:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c36ca55b800/2, rc 0, transno 4294967911 00000100:00080000:1.0:1789762868.232465:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762868.232467:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca55b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074700672:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1789762868.232484:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074700928 00000100:00100000:1.0:1789762868.232489:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074700928 00000100:00100000:1.0:1789762868.232523:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 331 00000100:00100000:1.0:1789762868.232531:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:7224:x1876702074700928:12345-0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1789762868.232722:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765e3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.232732:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765e3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.232738:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765e3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.232744:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762868.232748:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765e3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.232754:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765e3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.232773:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765e3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762868.232779:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074701056:12345-0@lo:6:osp-syn-1-1.0 Request processed in 935us (1275us total) trans 4294967399 rc 0/0 00010000:00010000:0.0:1789762868.232794:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1765e3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762868.232794:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000100:00100000:0.0:1789762868.232856:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067446912:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2951us (46363us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762868.232864:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 612 00000100:00100000:0.0:1789762868.232872:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341476889836 time 341476958059 state 0x2 ref 0 00000100:00100000:3.0:1789762868.232877:0:3425:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294967398 gen 2 00000004:00080000:3.0:1789762868.232883:0:3425:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37d9255500, transno 4294967398 00000100:00100000:3.0:1789762868.232895:0:3425:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9c36c9ffc380 x1876702074701056/t4294967399(4294967399) o6->lustre-OST0001-osc-MDT0001@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789762884 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-1.0' uid:0 gid:0 projid:4294967295 00000004:00080000:3.0:1789762868.232902:0:3425:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c36c9ffc380/2, rc 0, transno 4294967399 00000100:00100000:3.0:1789762868.232907:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074701056:0@lo:6:osp-syn-1-1.0 00000100:00100000:1.0:1789762868.232946:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074700928:12345-0@lo:1000:osp_up0-1.0 Request processed in 416us (1542us total) trans 4294967416 rc 0/0 00000100:00100000:1.0:1789762868.232957:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 331 00000100:00100000:0.0:1789762868.232974:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca559f80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074700928:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1789762868.232988:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 23 rpc_version 23 00010000:00010000:1.0:1789762868.232989:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00100000:0.0:1789762868.233001:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762868.233007:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c36cbac4000 x1876702074701312/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762868.233018:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac4000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074701312:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762868.233041:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762868.233058:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c36c8976540 going to sleep for 11 seconds 00000100:00100000:1.0:1789762868.233064:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074701312 00000100:00100000:1.0:1789762868.233069:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074701312 00000100:00100000:1.0:1789762868.233095:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:1.0:1789762868.233103:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074701312:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1789762868.233202:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7224:x1876702074701312:12345-0@lo:1000:osp_up0-1.0 Request processed in 100us (159us total) trans 4294967417 rc 0/0 00000100:00100000:1.0:1789762868.233208:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00100000:0.0:1789762868.233227:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac4000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074701312:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762868.234771:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762868.234804:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067447040 00000100:00100000:3.0:1789762868.234809:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067447040 00000100:00100000:3.0:1789762868.234833:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341476889836 time 341476958059 state 0x0 ref 1 00000100:00100000:3.0:1789762868.234842:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341526958059 time 341476958059 state 0x1 ref 1 00000100:00100000:2.0:1789762868.279964:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341526958059 time 341527035588 state 0x0 ref 1 00000100:00100000:2.0:1789762868.279980:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067447040:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762868.280002:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1765f8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762868.280020:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762868.280023:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762868.280027:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1765f8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762868.280036:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1765f8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.282419:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1765f8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.282434:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1765f8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.282444:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1765f8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.282452:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762868.282457:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1765f8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.282467:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1765f8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.282483:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1765f8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.282489:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000235a2c25/0xdf5a7ac59f1765f8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762868.282536:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067447040:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2560us (47766us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762868.282544:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 613 00000100:00100000:1.0:1789762868.282551:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341526958059 time 341527035588 state 0x2 ref 0 00000100:00100000:2.0:1789762868.284465:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.284511:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067447168 00000100:00100000:0.0:1789762868.284518:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067447168 00000100:00100000:0.0:1789762868.284549:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341526958059 time 341527035588 state 0x0 ref 1 00000100:00100000:0.0:1789762868.284568:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341577035588 time 341527035588 state 0x1 ref 1 00000100:00100000:0.0:1789762868.330011:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341577035588 time 341577081919 state 0x0 ref 1 00000100:00100000:0.0:1789762868.330026:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067447168:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762868.330049:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765ff lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.330064:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.330068:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.330075:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765ff lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.330088:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765ff lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.332435:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765ff lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.332446:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765ff lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.332453:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765ff lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.332470:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d10ba000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762868.332474:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765ff lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.332481:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765ff lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.332494:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765ff lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.332499:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f1765ff lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762868.332545:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067447168:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2522us (48081us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762868.332553:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 614 00000100:00100000:0.0:1789762868.332560:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341577035588 time 341577081919 state 0x2 ref 0 00000100:00100000:3.0:1789762868.334563:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.334596:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067447296 00000100:00100000:0.0:1789762868.334603:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067447296 00000100:00100000:0.0:1789762868.334633:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341577035588 time 341577081919 state 0x0 ref 1 00000100:00100000:0.0:1789762868.334642:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341627081919 time 341577081919 state 0x1 ref 1 00000100:00100000:0.0:1789762868.380063:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341627081919 time 341627130515 state 0x0 ref 1 00000100:00100000:0.0:1789762868.380092:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067447296:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762868.380119:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f176606 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.380130:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.380133:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.380137:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f176606 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.380146:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f176606 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.382525:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f176606 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.382536:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f176606 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.382542:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f176606 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.382547:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d10ba000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762868.382550:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f176606 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.382555:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f176606 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.382576:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f176606 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.382582:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009729f485/0xdf5a7ac59f176606 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762868.382626:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067447296:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2540us (48064us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762868.382632:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 615 00000100:00100000:0.0:1789762868.382639:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341627081919 time 341627130515 state 0x2 ref 0 00000100:00100000:3.0:1789762868.384433:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762868.384475:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067447424 00000100:00100000:1.0:1789762868.384482:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067447424 00000100:00100000:1.0:1789762868.384516:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341627081919 time 341627130515 state 0x0 ref 1 00000100:00100000:1.0:1789762868.384525:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341677130515 time 341627130515 state 0x1 ref 1 00000100:00100000:1.0:1789762868.430102:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341677130515 time 341677172164 state 0x0 ref 1 00000100:00100000:1.0:1789762868.430120:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067447424:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762868.430143:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f17660d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.430156:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762868.430159:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762868.430163:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f17660d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.430177:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f17660d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.432464:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f17660d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.432473:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f17660d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.432478:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f17660d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.432483:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5861400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762868.432487:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f17660d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.432493:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f17660d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.432507:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f17660d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.432513:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f17660d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762868.432559:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067447424:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2446us (48127us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762868.432566:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 616 00000100:00100000:1.0:1789762868.432573:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341677130515 time 341677172164 state 0x2 ref 0 00000100:00100000:3.0:1789762868.434182:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.434222:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067447552 00000100:00100000:0.0:1789762868.434229:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067447552 00000100:00100000:0.0:1789762868.434259:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341677130515 time 341677172164 state 0x0 ref 1 00000100:00100000:0.0:1789762868.434268:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341727172164 time 341677172164 state 0x1 ref 1 00000100:00100000:3.0:1789762868.480169:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341727172164 time 341727236572 state 0x0 ref 1 00000100:00100000:3.0:1789762868.480188:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067447552:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762868.480219:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176614 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.480254:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.480257:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.480262:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176614 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.480273:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176614 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.482787:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176614 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.482796:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176614 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.482803:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176614 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.482809:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee801e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762868.482812:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176614 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.482819:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176614 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.482833:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176614 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.482848:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176614 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762868.482891:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9fff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067447552:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2708us (48709us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762868.482898:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 617 00000100:00100000:3.0:1789762868.482905:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341727172164 time 341727236572 state 0x2 ref 0 00000100:00100000:3.0:1789762868.484678:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.484720:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067447680 00000100:00100000:0.0:1789762868.484728:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067447680 00000100:00100000:0.0:1789762868.484763:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341727172164 time 341727236572 state 0x0 ref 1 00000100:00100000:0.0:1789762868.484773:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341777236572 time 341727236572 state 0x1 ref 1 00000100:00100000:0.0:1789762868.530214:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341777236572 time 341777285702 state 0x0 ref 1 00000100:00100000:0.0:1789762868.530236:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067447680:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762868.530262:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17661b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.530273:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.530276:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.530280:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17661b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.530289:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17661b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.532679:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17661b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.532689:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17661b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.532695:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17661b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.532701:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762868.532704:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17661b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.532710:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17661b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.532731:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17661b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.532737:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f17661b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762868.532780:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067447680:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2546us (48102us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762868.532788:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 618 00000100:00100000:0.0:1789762868.532795:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341777236572 time 341777285702 state 0x2 ref 0 00000100:00100000:3.0:1789762868.534472:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762868.534517:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067447808 00000100:00100000:1.0:1789762868.534524:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067447808 00000100:00100000:1.0:1789762868.534554:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341777236572 time 341777285702 state 0x0 ref 1 00000100:00100000:1.0:1789762868.534566:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341827285702 time 341777285702 state 0x1 ref 1 00000100:00100000:3.0:1789762868.580289:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341827285702 time 341827346675 state 0x0 ref 1 00000100:00100000:3.0:1789762868.580316:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067447808:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762868.580363:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176622 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.580381:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.580386:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.580393:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176622 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.580408:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176622 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.583059:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176622 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.583069:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176622 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.583076:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176622 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.583081:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee801e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762868.583085:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176622 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.583092:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176622 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.583105:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176622 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.583110:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c75624b/0xdf5a7ac59f176622 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762868.583153:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067447808:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2846us (48683us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762868.583159:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 619 00000100:00100000:3.0:1789762868.583173:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341827285702 time 341827346675 state 0x2 ref 0 00000100:00100000:3.0:1789762868.584940:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.584988:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067447936 00000100:00100000:0.0:1789762868.584997:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067447936 00000100:00100000:0.0:1789762868.585032:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341827285702 time 341827346675 state 0x0 ref 1 00000100:00100000:0.0:1789762868.585042:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341877346675 time 341827346675 state 0x1 ref 1 00000100:00100000:1.0:1789762868.630383:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341877346675 time 341877443574 state 0x0 ref 1 00000100:00100000:1.0:1789762868.630404:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067447936:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762868.630433:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176629 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.630449:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762868.630453:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762868.630457:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176629 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.630466:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176629 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.633195:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176629 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.633206:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176629 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.633213:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176629 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.633218:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762868.633221:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176629 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.633227:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176629 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.633239:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176629 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.633244:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176629 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762868.633292:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067447936:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2896us (48351us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762868.633302:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 620 00000100:00100000:1.0:1789762868.633312:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341877346675 time 341877443574 state 0x2 ref 0 00000100:00100000:3.0:1789762868.635063:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.635115:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067448064 00000100:00100000:0.0:1789762868.635122:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067448064 00000100:00100000:0.0:1789762868.635152:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341877346675 time 341877443574 state 0x0 ref 1 00000100:00100000:0.0:1789762868.635162:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341927443574 time 341877443574 state 0x1 ref 1 00000100:00100000:2.0:1789762868.679854:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762868.679882:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762868.679899:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762868.679953:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067448320 00000100:00100000:0.0:1789762868.679955:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067448192 00000100:00100000:3.0:1789762868.679962:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067448320 00000100:00100000:2.0:1789762868.679971:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.679974:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067448192 00000100:00100000:1.0:1789762868.679979:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067448448 00000100:00100000:1.0:1789762868.679986:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067448448 00000100:00100000:2.0:1789762868.680003:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762868.680025:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067448704 00000100:00100000:3.0:1789762868.680030:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067448704 00000100:00100000:0.0:1789762868.680043:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 359 00000100:00100000:1.0:1789762868.680049:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 488 00000100:00100000:2.0:1789762868.680051:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.680057:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1551050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067448192:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762868.680064:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca558000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2395:x1876702067448320:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762868.680070:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067448832 00000100:00100000:3.0:1789762868.680074:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067448832 00000100:00100000:2.0:1789762868.680084:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 489 00000100:00100000:2.0:1789762868.680106:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067448448:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762868.680113:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 490 00000100:00100000:3.0:1789762868.680127:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067448704:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762868.680131:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1551050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067448192:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 73us (274us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762868.680133:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca558000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2395:x1876702067448320:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 67us (241us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762868.680143:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067448448:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 39us (240us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762868.680149:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 359 00000100:00100000:1.0:1789762868.680155:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 488 00000100:00100000:2.0:1789762868.680162:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 489 00000100:00100000:3.0:1789762868.680166:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067448704:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 44us (162us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762868.680178:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 490 00000100:00100000:1.0:1789762868.680187:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 491 00000100:00100000:0.0:1789762868.680195:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067448576 00000100:00100000:1.0:1789762868.680200:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067448832:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762868.680201:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067448576 00000100:00100000:1.0:1789762868.680236:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067448832:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 40us (187us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762868.680242:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 197 00000100:00100000:1.0:1789762868.680246:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 491 00000100:00080000:3.0:1789762868.680248:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789762868.680254:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762868.680255:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067448576:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:3.0:1789762868.680259:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762868.680286:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067448576:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 35us (316us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762868.680299:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 197 00000100:00100000:0.0:1789762868.680415:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341927443574 time 341927485808 state 0x0 ref 1 00000100:00100000:0.0:1789762868.680430:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067448064:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762868.680455:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000049855c25/0xdf5a7ac59f176630 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.680469:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.680472:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.680476:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000049855c25/0xdf5a7ac59f176630 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.680486:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000049855c25/0xdf5a7ac59f176630 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.682899:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000049855c25/0xdf5a7ac59f176630 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.682911:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000049855c25/0xdf5a7ac59f176630 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.682947:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000049855c25/0xdf5a7ac59f176630 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.682953:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9345600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762868.682957:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000049855c25/0xdf5a7ac59f176630 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.682964:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000049855c25/0xdf5a7ac59f176630 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.682978:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000049855c25/0xdf5a7ac59f176630 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.682983:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000049855c25/0xdf5a7ac59f176630 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762868.683035:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067448064:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2608us (47973us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762868.683041:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 621 00000100:00100000:0.0:1789762868.683054:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341927443574 time 341927485808 state 0x2 ref 0 00000100:00100000:0.0:1789762868.684987:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762868.685023:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067448960 00000100:00100000:3.0:1789762868.685028:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067448960 00000100:00100000:3.0:1789762868.685054:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341927443574 time 341927485808 state 0x0 ref 1 00000100:00100000:3.0:1789762868.685073:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341977485808 time 341927485808 state 0x1 ref 1 00000100:00100000:3.0:1789762868.730471:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341977485808 time 341977538713 state 0x0 ref 1 00000100:00100000:3.0:1789762868.730490:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067448960:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762868.730523:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176637 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.730535:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.730538:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.730542:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176637 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.730551:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176637 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.732756:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176637 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.732764:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176637 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.732770:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176637 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.732775:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762868.732778:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176637 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.732784:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176637 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.732806:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176637 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.732812:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176637 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762868.732852:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067448960:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2367us (47866us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762868.732858:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 622 00000100:00100000:3.0:1789762868.732865:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341977485808 time 341977538713 state 0x2 ref 0 00000100:00100000:3.0:1789762868.734532:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762868.734575:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067449088 00000100:00100000:1.0:1789762868.734582:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067449088 00000100:00100000:1.0:1789762868.734612:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 341977485808 time 341977538713 state 0x0 ref 1 00000100:00100000:1.0:1789762868.734621:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342027538713 time 341977538713 state 0x1 ref 1 00000100:00100000:1.0:1789762868.780519:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342027538713 time 342027589818 state 0x0 ref 1 00000100:00100000:1.0:1789762868.780536:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067449088:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762868.780560:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17663e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.780573:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762868.780576:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762868.780580:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17663e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.780590:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17663e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.782850:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17663e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.782861:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17663e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.782869:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17663e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.782874:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762868.782878:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17663e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.782884:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17663e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.782896:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17663e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.782906:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17663e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762868.782975:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067449088:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2444us (48442us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762868.782982:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 623 00000100:00100000:1.0:1789762868.782989:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342027538713 time 342027589818 state 0x2 ref 0 00000100:00100000:3.0:1789762868.784890:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.784954:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067449216 00000100:00100000:0.0:1789762868.784961:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067449216 00000100:00100000:0.0:1789762868.784993:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342027538713 time 342027589818 state 0x0 ref 1 00000100:00100000:0.0:1789762868.785002:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342077589818 time 342027589818 state 0x1 ref 1 00000100:00100000:0.0:1789762868.830572:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342077589818 time 342077643134 state 0x0 ref 1 00000100:00100000:0.0:1789762868.830588:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067449216:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762868.830612:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004b01b0ef/0xdf5a7ac59f176645 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.830623:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.830626:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.830631:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b01b0ef/0xdf5a7ac59f176645 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.830639:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b01b0ef/0xdf5a7ac59f176645 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.833217:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004b01b0ef/0xdf5a7ac59f176645 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.833232:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b01b0ef/0xdf5a7ac59f176645 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.833249:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b01b0ef/0xdf5a7ac59f176645 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.833257:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee800600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762868.833262:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b01b0ef/0xdf5a7ac59f176645 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.833288:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b01b0ef/0xdf5a7ac59f176645 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.833306:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b01b0ef/0xdf5a7ac59f176645 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.833315:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b01b0ef/0xdf5a7ac59f176645 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762868.833377:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067449216:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2792us (48487us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762868.833384:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 624 00000100:00100000:3.0:1789762868.833392:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342077589818 time 342077643134 state 0x2 ref 0 00000100:00100000:3.0:1789762868.835234:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.835272:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067449344 00000100:00100000:0.0:1789762868.835279:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067449344 00000100:00100000:0.0:1789762868.835312:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342077589818 time 342077643134 state 0x0 ref 1 00000100:00100000:0.0:1789762868.835348:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342127643134 time 342077643134 state 0x1 ref 1 00010000:00010000:2.0:1789762868.863997:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762868.880617:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342127643134 time 342127688918 state 0x0 ref 1 00000100:00100000:0.0:1789762868.880634:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067449344:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762868.880666:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000dcbdc4da/0xdf5a7ac59f17664c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.880684:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.880689:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762868.880694:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcbdc4da/0xdf5a7ac59f17664c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.880704:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcbdc4da/0xdf5a7ac59f17664c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.883353:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000dcbdc4da/0xdf5a7ac59f17664c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.883364:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcbdc4da/0xdf5a7ac59f17664c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.883370:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcbdc4da/0xdf5a7ac59f17664c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.883389:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee801800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762868.883393:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcbdc4da/0xdf5a7ac59f17664c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.883400:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcbdc4da/0xdf5a7ac59f17664c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.883414:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcbdc4da/0xdf5a7ac59f17664c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762868.883419:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcbdc4da/0xdf5a7ac59f17664c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762868.883464:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067449344:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2835us (48230us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762868.883471:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 625 00000100:00100000:0.0:1789762868.883478:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342127643134 time 342127688918 state 0x2 ref 0 00000100:00100000:3.0:1789762868.885164:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762868.885200:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067449472 00000100:00100000:1.0:1789762868.885208:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067449472 00000100:00100000:1.0:1789762868.885243:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342127643134 time 342127688918 state 0x0 ref 1 00000100:00100000:1.0:1789762868.885258:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342177688918 time 342127688918 state 0x1 ref 1 00000100:00100000:3.0:1789762868.930780:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342177688918 time 342177837128 state 0x0 ref 1 00000100:00100000:3.0:1789762868.930819:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067449472:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762868.930879:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176653 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.930896:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.930899:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762868.930914:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176653 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.930925:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176653 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.935086:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176653 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.935097:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176653 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.935104:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176653 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.935133:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762868.935140:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176653 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.935151:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176653 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.935177:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176653 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762868.935188:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176653 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762868.935297:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067449472:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4490us (50129us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762868.935310:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 626 00000100:00100000:3.0:1789762868.935322:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342177688918 time 342177837128 state 0x2 ref 0 00000100:00100000:3.0:1789762868.938525:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.938628:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067449600 00000100:00100000:0.0:1789762868.938642:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067449600 00000100:00100000:0.0:1789762868.938699:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342177688918 time 342177837128 state 0x0 ref 1 00000100:00100000:0.0:1789762868.938718:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342227837128 time 342177837128 state 0x1 ref 1 00000100:00100000:1.0:1789762868.980955:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342227837128 time 342227979637 state 0x0 ref 1 00000100:00100000:1.0:1789762868.981001:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067449600:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762868.981042:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17665a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.981059:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762868.981063:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762868.981069:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17665a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.981090:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17665a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.985310:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17665a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.985347:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17665a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.985357:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17665a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.985381:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762868.985386:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17665a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.985394:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17665a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.985413:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17665a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762868.985421:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17665a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762868.985492:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067449600:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4514us (46970us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762868.985502:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 627 00000100:00100000:1.0:1789762868.985513:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342227837128 time 342227979637 state 0x2 ref 0 00000100:00100000:3.0:1789762868.988714:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762868.988809:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067449728 00000100:00100000:0.0:1789762868.988847:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067449728 00000100:00100000:0.0:1789762868.988921:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342227837128 time 342227979637 state 0x0 ref 1 00000100:00100000:0.0:1789762868.988936:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342277979637 time 342227979637 state 0x1 ref 1 00000100:00100000:0.0:1789762869.031044:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342277979637 time 342278111355 state 0x0 ref 1 00000100:00100000:0.0:1789762869.031083:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067449728:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:3.0:1789762869.031118:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762869.031119:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176661 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.031139:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.031144:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.031151:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176661 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.031163:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176661 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.035951:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176661 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.035967:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176661 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.035977:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176661 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.035997:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee800800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762869.036003:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176661 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.036012:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176661 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.036030:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176661 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.036038:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176661 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762869.036099:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067449728:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5038us (47389us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762869.036109:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 628 00000100:00100000:0.0:1789762869.036119:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342277979637 time 342278111355 state 0x2 ref 0 00000100:00100000:3.0:1789762869.039472:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762869.039526:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067449856 00000100:00100000:0.0:1789762869.039536:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067449856 00000100:00100000:0.0:1789762869.039598:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342277979637 time 342278111355 state 0x0 ref 1 00000100:00100000:0.0:1789762869.039617:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342328111355 time 342278111355 state 0x1 ref 1 00000100:00100000:3.0:1789762869.081162:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342328111355 time 342328225634 state 0x0 ref 1 00000100:00100000:3.0:1789762869.081187:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067449856:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762869.081227:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176668 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.081263:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.081271:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.081282:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176668 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.081306:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176668 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.085945:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176668 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.085974:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176668 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.085987:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176668 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.085999:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762869.086005:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176668 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.086019:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176668 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.086045:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176668 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.086058:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f176668 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762869.086136:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067449856:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4954us (46665us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762869.086149:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 629 00000100:00100000:0.0:1789762869.086161:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342328111355 time 342328225634 state 0x2 ref 0 00000100:00100000:3.0:1789762869.089798:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762869.089870:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067449984 00000100:00100000:1.0:1789762869.089882:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067449984 00000100:00100000:1.0:1789762869.089997:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342328111355 time 342328225634 state 0x0 ref 1 00000100:00100000:1.0:1789762869.090024:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342378225634 time 342328225634 state 0x1 ref 1 00000100:00100000:1.0:1789762869.131236:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342378225634 time 342378299371 state 0x0 ref 1 00000100:00100000:1.0:1789762869.131263:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067449984:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762869.131302:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17666f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.131322:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762869.131350:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762869.131358:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17666f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.131390:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17666f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.136185:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17666f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.136213:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17666f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.136237:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17666f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.136254:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762869.136263:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17666f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.136281:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17666f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.136304:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17666f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.136345:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17666f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762869.136425:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067449984:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5168us (46630us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762869.136439:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 630 00000100:00100000:1.0:1789762869.136475:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342378225634 time 342378299371 state 0x2 ref 0 00000100:00100000:3.0:1789762869.139617:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762869.139714:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067450112 00000100:00100000:0.0:1789762869.139735:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067450112 00000100:00100000:0.0:1789762869.139783:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342378225634 time 342378299371 state 0x0 ref 1 00000100:00100000:0.0:1789762869.139800:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342428299371 time 342378299371 state 0x1 ref 1 00000100:00100000:3.0:1789762869.181433:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342428299371 time 342428496012 state 0x0 ref 1 00000100:00100000:3.0:1789762869.181466:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067450112:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762869.181512:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176676 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.181539:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.181547:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.181559:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176676 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.181583:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176676 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.186220:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176676 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.186237:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176676 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.186249:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176676 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.186257:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762869.186262:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176676 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.186285:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176676 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.186306:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176676 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.186316:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176676 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762869.186418:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067450112:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4960us (46802us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762869.186430:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 631 00000100:00100000:3.0:1789762869.186441:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342428299371 time 342428496012 state 0x2 ref 0 00000100:00100000:3.0:1789762869.189791:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762869.189945:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067450240 00000100:00100000:0.0:1789762869.189958:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067450240 00000100:00100000:0.0:1789762869.190014:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342428299371 time 342428496012 state 0x0 ref 1 00000100:00100000:0.0:1789762869.190031:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342478496012 time 342428496012 state 0x1 ref 1 00000100:00100000:0.0:1789762869.231514:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342478496012 time 342478576953 state 0x0 ref 1 00000100:00100000:0.0:1789762869.231540:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067450240:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762869.231580:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f17667d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.231599:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.231605:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.231612:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f17667d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.231627:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f17667d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.236578:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f17667d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.236611:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f17667d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.236625:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f17667d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.236636:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee800800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762869.236642:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f17667d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.236653:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f17667d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.236677:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f17667d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.236687:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f17667d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762869.236772:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067450240:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5237us (46982us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762869.236784:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 632 00000100:00100000:0.0:1789762869.236796:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342478496012 time 342478576953 state 0x2 ref 0 00000100:00100000:3.0:1789762869.240194:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762869.240277:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067450368 00000100:00100000:1.0:1789762869.240289:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067450368 00000100:00100000:1.0:1789762869.240375:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342478496012 time 342478576953 state 0x0 ref 1 00000100:00100000:1.0:1789762869.240393:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342528576953 time 342478576953 state 0x1 ref 1 00000100:00100000:3.0:1789762869.281649:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342528576953 time 342528713330 state 0x0 ref 1 00000100:00100000:3.0:1789762869.281675:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067450368:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762869.281712:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176684 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.281730:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.281736:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.281757:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176684 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.281789:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176684 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.286263:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176684 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.286283:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176684 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.286295:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176684 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.286305:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762869.286310:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176684 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.286320:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176684 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.286373:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176684 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.286390:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176684 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762869.286515:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067450368:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4841us (46318us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762869.286533:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 633 00000100:00100000:3.0:1789762869.286551:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342528576953 time 342528713330 state 0x2 ref 0 00000100:00100000:3.0:1789762869.289409:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762869.289489:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067450496 00000100:00100000:0.0:1789762869.289514:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067450496 00000100:00100000:0.0:1789762869.289574:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342528576953 time 342528713330 state 0x0 ref 1 00000100:00100000:0.0:1789762869.289601:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342578713330 time 342528713330 state 0x1 ref 1 00000100:00100000:1.0:1789762869.331766:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342578713330 time 342578830345 state 0x0 ref 1 00000100:00100000:1.0:1789762869.331793:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067450496:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762869.331829:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17668b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.331862:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762869.331868:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762869.331875:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17668b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.331891:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17668b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.336584:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17668b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.336613:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17668b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.336634:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17668b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.336651:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762869.336661:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17668b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.336683:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17668b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.336717:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17668b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.336736:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17668b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762869.336834:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067450496:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5044us (47422us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762869.336851:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 634 00000100:00100000:1.0:1789762869.336866:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342578713330 time 342578830345 state 0x2 ref 0 00000100:00100000:3.0:1789762869.340304:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762869.340408:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067450624 00000100:00100000:0.0:1789762869.340422:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067450624 00000100:00100000:0.0:1789762869.340483:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342578713330 time 342578830345 state 0x0 ref 1 00000100:00100000:0.0:1789762869.340501:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342628830345 time 342578830345 state 0x1 ref 1 00000100:00100000:0.0:1789762869.381903:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342628830345 time 342628962202 state 0x0 ref 1 00000100:00100000:0.0:1789762869.381979:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067450624:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762869.382028:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176692 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.382052:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.382058:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.382066:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176692 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.382085:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176692 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.387102:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176692 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.387141:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176692 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.387154:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176692 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.387164:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee800800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762869.387169:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176692 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.387180:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176692 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.387202:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176692 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.387212:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f176692 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762869.387297:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067450624:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5327us (46995us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762869.387310:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 635 00000100:00100000:0.0:1789762869.387418:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342628830345 time 342628962202 state 0x2 ref 0 00000100:00100000:3.0:1789762869.390801:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762869.390953:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067450752 00000100:00100000:0.0:1789762869.390979:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067450752 00000100:00100000:0.0:1789762869.391034:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342628830345 time 342628962202 state 0x0 ref 1 00000100:00100000:0.0:1789762869.391053:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342678962202 time 342628962202 state 0x1 ref 1 00000100:00100000:3.0:1789762869.432067:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342678962202 time 342679131717 state 0x0 ref 1 00000100:00100000:3.0:1789762869.432091:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067450752:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762869.432126:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176699 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.432144:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.432149:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.432156:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176699 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.432170:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176699 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.436516:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176699 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.436548:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176699 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.436558:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176699 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.436566:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762869.436571:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176699 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.436581:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176699 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.436599:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176699 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.436607:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176699 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762869.436670:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067450752:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4585us (45873us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762869.436680:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 636 00000100:00100000:3.0:1789762869.436689:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342678962202 time 342679131717 state 0x2 ref 0 00000100:00100000:3.0:1789762869.439675:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762869.439745:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067450880 00000100:00100000:1.0:1789762869.439755:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067450880 00000100:00100000:1.0:1789762869.439804:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342678962202 time 342679131717 state 0x0 ref 1 00000100:00100000:1.0:1789762869.439823:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342729131717 time 342679131717 state 0x1 ref 1 00000100:00100000:1.0:1789762869.482185:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342729131717 time 342729242168 state 0x0 ref 1 00000100:00100000:1.0:1789762869.482220:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067450880:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762869.482272:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766a0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.482296:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762869.482302:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762869.482311:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766a0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.482357:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766a0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.486860:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766a0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.486876:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766a0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.486887:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766a0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.486901:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762869.486906:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766a0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.486960:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766a0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.486984:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766a0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.486995:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766a0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762869.487079:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067450880:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4868us (47403us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762869.487091:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 637 00000100:00100000:1.0:1789762869.487104:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342729131717 time 342729242168 state 0x2 ref 0 00000100:00100000:3.0:1789762869.490195:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762869.490349:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067451008 00000100:00100000:0.0:1789762869.490360:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067451008 00000100:00100000:0.0:1789762869.490410:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342729131717 time 342729242168 state 0x0 ref 1 00000100:00100000:0.0:1789762869.490426:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342779242168 time 342729242168 state 0x1 ref 1 00000100:00100000:3.0:1789762869.532294:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342779242168 time 342779361543 state 0x0 ref 1 00000100:00100000:3.0:1789762869.532315:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a09c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067451008:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762869.532367:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f1766a7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.532382:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.532385:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.532391:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f1766a7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.532401:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f1766a7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.535271:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f1766a7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.535284:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f1766a7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.535292:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f1766a7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.535298:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762869.535302:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f1766a7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.535308:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f1766a7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.535358:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f1766a7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.535382:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f1766a7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762869.535435:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a09c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067451008:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3125us (45245us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762869.535442:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 638 00000100:00100000:0.0:1789762869.535449:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342779242168 time 342779361543 state 0x2 ref 0 00000100:00100000:2.0:1789762869.537401:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762869.537451:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067451136 00000100:00100000:3.0:1789762869.537460:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067451136 00000100:00100000:3.0:1789762869.537502:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342779242168 time 342779361543 state 0x0 ref 1 00000100:00100000:3.0:1789762869.537513:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342829361543 time 342779361543 state 0x1 ref 1 00000100:00100000:0.0:1789762869.582397:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342829361543 time 342829466764 state 0x0 ref 1 00000100:00100000:0.0:1789762869.582415:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067451136:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762869.582446:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f1766ae lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.582459:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.582462:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.582467:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f1766ae lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.582476:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f1766ae lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.585249:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f1766ae lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.585260:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f1766ae lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.585278:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f1766ae lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.585284:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee800800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762869.585287:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f1766ae lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.585293:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f1766ae lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.585305:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f1766ae lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.585311:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b4409e0/0xdf5a7ac59f1766ae lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762869.585378:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067451136:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2966us (47976us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762869.585389:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 639 00000100:00100000:3.0:1789762869.585396:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342829361543 time 342829466764 state 0x2 ref 0 00000100:00100000:2.0:1789762869.587162:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762869.587210:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067451264 00000100:00100000:1.0:1789762869.587217:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067451264 00000100:00100000:1.0:1789762869.587253:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342829361543 time 342829466764 state 0x0 ref 1 00000100:00100000:1.0:1789762869.587263:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342879466764 time 342829466764 state 0x1 ref 1 00000100:00100000:3.0:1789762869.632517:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342879466764 time 342879582193 state 0x0 ref 1 00000100:00100000:3.0:1789762869.632538:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067451264:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762869.632582:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766b5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.632597:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.632600:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.632605:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766b5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.632616:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766b5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.635386:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766b5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.635396:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766b5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.635403:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766b5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.635408:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762869.635411:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766b5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.635417:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766b5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.635442:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766b5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.635448:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766b5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762869.635495:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067451264:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2964us (48334us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762869.635502:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 640 00000100:00100000:3.0:1789762869.635509:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342879466764 time 342879582193 state 0x2 ref 0 00000100:00100000:0.0:1789762869.637516:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762869.637551:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067451392 00000100:00100000:3.0:1789762869.637557:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067451392 00000100:00100000:3.0:1789762869.637590:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342879466764 time 342879582193 state 0x0 ref 1 00000100:00100000:3.0:1789762869.637600:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342929582193 time 342879582193 state 0x1 ref 1 00000100:00100000:1.0:1789762869.682629:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342929582193 time 342929695667 state 0x0 ref 1 00000100:00100000:1.0:1789762869.682650:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067451392:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762869.682684:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766bc lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.682697:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762869.682700:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762869.682705:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766bc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.682716:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766bc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.685492:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766bc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.685503:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766bc lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.685510:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766bc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.685515:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762869.685518:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766bc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.685525:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766bc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.685538:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766bc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.685543:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766bc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762869.685591:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067451392:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2948us (48076us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762869.685598:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 641 00000100:00100000:1.0:1789762869.685605:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342929582193 time 342929695667 state 0x2 ref 0 00000100:00100000:0.0:1789762869.687578:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762869.687614:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067451520 00000100:00100000:3.0:1789762869.687622:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067451520 00000100:00100000:3.0:1789762869.687658:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342929582193 time 342929695667 state 0x0 ref 1 00000100:00100000:3.0:1789762869.687669:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342979695667 time 342929695667 state 0x1 ref 1 00000100:00100000:0.0:1789762869.732715:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342979695667 time 342979783010 state 0x0 ref 1 00000100:00100000:0.0:1789762869.732733:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067451520:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762869.732781:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766c3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.732798:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.732801:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.732806:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766c3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.732817:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766c3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.735427:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766c3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.735436:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766c3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.735441:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766c3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.735456:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762869.735460:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766c3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.735466:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766c3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.735480:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766c3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.735486:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766c3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762869.735532:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067451520:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2804us (47955us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762869.735540:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 642 00000100:00100000:0.0:1789762869.735547:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342979695667 time 342979783010 state 0x2 ref 0 00000100:00100000:0.0:1789762869.737518:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762869.737560:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067451648 00000100:00100000:3.0:1789762869.737574:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067451648 00000100:00100000:3.0:1789762869.737608:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 342979695667 time 342979783010 state 0x0 ref 1 00000100:00100000:3.0:1789762869.737619:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343029783010 time 342979783010 state 0x1 ref 1 00000100:00100000:0.0:1789762869.760028:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074701440:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762869.760079:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762869.760093:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074701568:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762869.760141:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c992bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074701952:0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.760142:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762869.760181:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c992b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074702080:0@lo:400:kworker.0 00000100:00100000:0.0:1789762869.760191:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762869.760225:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762869.760235:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f60a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074702464:0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.760245:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074701440 00000100:00100000:2.0:1789762869.760252:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f60a0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074702592:0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.760254:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074701440 00000100:00100000:0.0:1789762869.760292:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762869.760298:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762869.760302:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074701568 00000100:00100000:3.0:1789762869.760306:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074701568 00000100:00100000:0.0:1789762869.760342:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f60a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074702976:0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.760357:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f60a3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074703104:0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.760362:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:0.0:1789762869.760370:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762869.760377:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074701440:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762869.760398:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762869.760401:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1789762869.760409:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7fbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074701824:0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762869.760415:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3851419648 avail 3849322496 00000100:00100000:2.0:1789762869.760432:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:2.0:1789762869.760441:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7fb8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074701696:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762869.760446:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762869.760471:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074701440:12345-0@lo:13:osp-pre-1-1.0 Request processed in 99us (392us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762869.760478:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9929180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074702336:0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.760481:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:2.0:1789762869.760483:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762869.760506:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074701824 00000100:00100000:2.0:1789762869.760510:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c992a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074702208:0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.760511:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074701824 00000100:00100000:0.0:1789762869.760520:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762869.760538:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074701696 00000100:00100000:3.0:1789762869.760542:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074701696 00000100:00100000:0.0:1789762869.760546:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f60a1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074702848:0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.760557:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762869.760568:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:3.0:1789762869.760576:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074701568:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762869.760584:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762869.760587:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f60a3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074702720:0@lo:400:kworker.0 00002000:00000024:3.0:1789762869.760589:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3851419648 avail 3849322496 00000100:00100000:2.0:1789762869.760634:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762869.760637:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074701440:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762869.760641:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074701568:12345-0@lo:13:osp-pre-1-0.0 Request processed in 65us (502us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762869.760650:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:2.0:1789762869.760666:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f60a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074703232:0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.760672:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:3.0:1789762869.760681:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074701824:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762869.760683:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074701952 00000100:00100000:0.0:1789762869.760688:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074701952 00000100:00100000:2.0:1789762869.760697:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:3.0:1789762869.760723:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3746562048 avail 3744464896 00000100:00100000:0.0:1789762869.760729:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1789762869.760742:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074701568:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762869.760745:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1552c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074701952:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.760777:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074702080 00000100:00100000:2.0:1789762869.760781:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074702080 00000100:00100000:3.0:1789762869.760797:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074701824:12345-0@lo:13:osp-pre-0-0.0 Request processed in 116us (354us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762869.760808:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1552c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074701952:12345-0@lo:400:kworker.0 Request processed in 69us (621us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762869.760812:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074702592 00000100:00100000:3.0:1789762869.760812:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:2.0:1789762869.760818:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074702592 00000100:00100000:0.0:1789762869.760821:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00100000:3.0:1789762869.760844:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:2.0:1789762869.760857:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074702976 00000100:00100000:3.0:1789762869.760858:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074701696:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762869.760861:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074702976 00000100:00100000:0.0:1789762869.760867:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074702336 00000100:00100000:0.0:1789762869.760872:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074702336 00002000:00000024:3.0:1789762869.760876:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3746562048 avail 3744464896 00000100:00100000:2.0:1789762869.760886:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074702848 00000100:00100000:2.0:1789762869.760891:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074702848 00000100:00100000:0.0:1789762869.760913:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074702720 00000100:00100000:0.0:1789762869.760935:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074702720 00000100:00100000:2.0:1789762869.760943:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 492 00000100:00100000:2.0:1789762869.760959:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074702080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.760960:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074701696:12345-0@lo:13:osp-pre-0-1.0 Request processed in 102us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762869.760966:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 493 00000100:00100000:3.0:1789762869.760979:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:0.0:1789762869.760982:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074702592:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789762869.761012:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762869.761014:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074702080:12345-0@lo:400:kworker.0 Request processed in 60us (788us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762869.761020:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762869.761027:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 492 00000100:00080000:3.0:1789762869.761027:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762869.761033:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074702592:12345-0@lo:400:kworker.0 Request processed in 55us (737us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762869.761036:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762869.761051:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 493 00000100:00100000:2.0:1789762869.761057:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 494 00000100:00100000:2.0:1789762869.761069:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074702976:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762869.761077:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 495 00000100:00100000:3.0:1789762869.761086:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 496 00000100:00100000:0.0:1789762869.761088:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074702336:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.761103:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074702848:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.761111:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074702976:12345-0@lo:400:kworker.0 Request processed in 45us (740us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762869.761117:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 494 00000100:00100000:2.0:1789762869.761142:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 497 00000100:00100000:0.0:1789762869.761143:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074702336:12345-0@lo:400:kworker.0 Request processed in 54us (621us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762869.761152:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 495 00000100:00100000:2.0:1789762869.761156:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074702720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.761164:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074702848:12345-0@lo:400:kworker.0 Request processed in 65us (579us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762869.761174:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 496 00000100:00100000:0.0:1789762869.761188:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074702464 00000100:00100000:0.0:1789762869.761193:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074702464 00000100:00100000:2.0:1789762869.761200:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074702720:12345-0@lo:400:kworker.0 Request processed in 50us (567us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762869.761208:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 497 00000100:00100000:3.0:1789762869.761222:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074703104 00000100:00100000:0.0:1789762869.761228:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074702208 00000100:00100000:3.0:1789762869.761229:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074703104 00000100:00100000:0.0:1789762869.761232:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074702208 00000100:00100000:2.0:1789762869.761239:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074703232 00000100:00100000:2.0:1789762869.761244:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074703232 00000100:00100000:0.0:1789762869.761268:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 333 00000100:00100000:0.0:1789762869.761279:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074702464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.761284:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:2.0:1789762869.761293:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:3.0:1789762869.761299:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074703104:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.761306:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074703232:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762869.761318:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074702464:12345-0@lo:400:kworker.0 Request processed in 42us (1029us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762869.761340:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 333 00000100:00100000:1.0:1789762869.761350:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:1.0:1789762869.761365:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3426:x1876702074702208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.761371:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074703232:12345-0@lo:400:kworker.0 Request processed in 70us (675us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762869.761382:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00100000:3.0:1789762869.761384:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074703104:12345-0@lo:400:kworker.0 Request processed in 86us (985us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762869.761396:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:0.0:1789762869.761399:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7fbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074701824:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762869.761417:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3426:x1876702074702208:12345-0@lo:400:kworker.0 Request processed in 56us (860us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762869.761421:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9929180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074702336:0@lo:400:kworker.0 00000100:00100000:1.0:1789762869.761428:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00100000:2.0:1789762869.761437:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7fb8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074701696:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762869.761442:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f60a1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074702848:0@lo:400:kworker.0 00000100:00100000:0.0:1789762869.761463:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f60a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074702464:0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.761473:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c992a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074702208:0@lo:400:kworker.0 00000100:00100000:0.0:1789762869.761483:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f60a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074702976:0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.761494:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f60a3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074702720:0@lo:400:kworker.0 00000100:00100000:0.0:1789762869.761498:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c992bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074701952:0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.761510:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f60a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074703232:0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.761525:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f60a3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074703104:0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.761534:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c992b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074702080:0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.761541:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f60a0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074702592:0@lo:400:kworker.0 00000100:00100000:0.0:1789762869.782808:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343029783010 time 343029876996 state 0x0 ref 1 00000100:00100000:0.0:1789762869.782833:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067451648:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762869.782859:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766ca lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.782872:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762869.782876:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.782881:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766ca lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.782892:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766ca lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.785441:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766ca lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.785452:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766ca lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.785458:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766ca lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.785463:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762869.785466:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766ca lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.785472:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766ca lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.785493:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766ca lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.785499:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766ca lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762869.785540:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067451648:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2710us (48023us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762869.785547:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 643 00000100:00100000:0.0:1789762869.785554:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343029783010 time 343029876996 state 0x2 ref 0 00000100:00100000:0.0:1789762869.787439:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762869.787490:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067451776 00000100:00100000:1.0:1789762869.787497:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067451776 00000100:00100000:1.0:1789762869.787532:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343029783010 time 343029876996 state 0x0 ref 1 00000100:00100000:1.0:1789762869.787542:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343079876996 time 343029876996 state 0x1 ref 1 00000100:00100000:1.0:1789762869.832869:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343079876996 time 343079933710 state 0x0 ref 1 00000100:00100000:1.0:1789762869.832895:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067451776:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762869.832944:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766d1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.832958:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762869.832962:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762869.832967:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766d1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.832978:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766d1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.835708:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766d1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.835718:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766d1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.835726:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766d1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.835732:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762869.835735:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766d1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.835742:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766d1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.835757:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766d1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762869.835762:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766d1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762869.835808:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067451776:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2921us (48369us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762869.835816:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 644 00000100:00100000:1.0:1789762869.835824:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343079876996 time 343079933710 state 0x2 ref 0 00000100:00100000:0.0:1789762869.838024:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762869.838081:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067451904 00000100:00100000:3.0:1789762869.838092:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067451904 00000100:00100000:3.0:1789762869.838136:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343079876996 time 343079933710 state 0x0 ref 1 00000100:00100000:3.0:1789762869.838151:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343129933710 time 343079933710 state 0x1 ref 1 00000100:00100000:0.0:1789762869.882999:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343129933710 time 343130064698 state 0x0 ref 1 00000100:00100000:0.0:1789762869.883019:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067451904:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762869.883055:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766d8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.883068:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.883072:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.883077:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766d8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.883104:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766d8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.885812:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766d8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.885823:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766d8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.885831:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766d8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.885836:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762869.885839:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766d8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.885845:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766d8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.885858:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766d8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.885863:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766d8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762869.885949:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067451904:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2936us (47925us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762869.885958:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 645 00000100:00100000:0.0:1789762869.885966:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343129933710 time 343130064698 state 0x2 ref 0 00000100:00100000:0.0:1789762869.887975:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762869.888020:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067452032 00000100:00100000:3.0:1789762869.888028:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067452032 00000100:00100000:3.0:1789762869.888065:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343129933710 time 343130064698 state 0x0 ref 1 00010000:00010000:1.0:1789762869.888076:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762869.888077:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343180064698 time 343130064698 state 0x1 ref 1 00000100:00100000:3.0:1789762869.933064:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343180064698 time 343180128058 state 0x0 ref 1 00000100:00100000:3.0:1789762869.933085:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067452032:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762869.933121:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766df lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.933136:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.933139:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762869.933144:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766df lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.933154:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766df lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.936007:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766df lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.936019:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766df lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.936027:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766df lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.936033:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762869.936037:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766df lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.936043:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766df lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.936058:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766df lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762869.936065:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f1766df lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762869.936112:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067452032:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3035us (48138us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762869.936119:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 646 00000100:00100000:3.0:1789762869.936132:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343180064698 time 343180128058 state 0x2 ref 0 00000100:00100000:2.0:1789762869.938092:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762869.938133:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067452160 00000100:00100000:1.0:1789762869.938140:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067452160 00000100:00100000:1.0:1789762869.938177:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343180064698 time 343180128058 state 0x0 ref 1 00000100:00100000:1.0:1789762869.938187:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343230128058 time 343180128058 state 0x1 ref 1 00000100:00100000:0.0:1789762869.983171:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343230128058 time 343230236338 state 0x0 ref 1 00000100:00100000:0.0:1789762869.983192:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067452160:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762869.983226:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766e6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.983239:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.983243:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762869.983247:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766e6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.983258:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766e6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.985955:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766e6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.985965:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766e6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.985982:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766e6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.985988:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762869.985991:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766e6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.985997:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766e6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.986010:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766e6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762869.986015:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f1766e6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762869.986058:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067452160:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2873us (47967us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762869.986065:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 647 00000100:00100000:0.0:1789762869.986072:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343230128058 time 343230236338 state 0x2 ref 0 00000100:00100000:2.0:1789762869.987983:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762869.988014:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067452288 00000100:00100000:0.0:1789762869.988022:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067452288 00000100:00100000:0.0:1789762869.988058:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343230128058 time 343230236338 state 0x0 ref 1 00000100:00100000:0.0:1789762869.988077:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343280236338 time 343230236338 state 0x1 ref 1 00000100:00100000:1.0:1789762870.033336:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343280236338 time 343280383788 state 0x0 ref 1 00000100:00100000:1.0:1789762870.033378:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067452288:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762870.033443:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766ed lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.033466:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762870.033471:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762870.033477:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766ed lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.033490:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766ed lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.036891:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766ed lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.036902:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766ed lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.036909:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766ed lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.036932:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762870.036935:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766ed lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.036942:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766ed lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.036958:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766ed lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.036968:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1766ed lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762870.037013:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067452288:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3644us (49031us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762870.037020:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 648 00000100:00100000:1.0:1789762870.037028:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343280236338 time 343280383788 state 0x2 ref 0 00000100:00100000:0.0:1789762870.039522:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762870.039569:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067452416 00000100:00100000:3.0:1789762870.039579:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067452416 00000100:00100000:3.0:1789762870.039636:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343280236338 time 343280383788 state 0x0 ref 1 00000100:00100000:3.0:1789762870.039647:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343330383788 time 343280383788 state 0x1 ref 1 00000100:00100000:3.0:1789762870.083402:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343330383788 time 343330467616 state 0x0 ref 1 00000100:00100000:3.0:1789762870.083422:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067452416:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762870.083456:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766f4 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.083471:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.083474:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.083480:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766f4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.083491:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766f4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.086125:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766f4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.086136:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766f4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.086156:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766f4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.086162:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f63ad400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762870.086165:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766f4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.086172:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766f4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.086185:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766f4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.086190:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766f4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762870.086238:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067452416:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2823us (46717us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762870.086247:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 649 00000100:00100000:3.0:1789762870.086265:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343330383788 time 343330467616 state 0x2 ref 0 00000100:00100000:3.0:1789762870.088397:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762870.088449:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067452544 00000100:00100000:0.0:1789762870.088457:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067452544 00000100:00100000:0.0:1789762870.088497:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343330383788 time 343330467616 state 0x0 ref 1 00000100:00100000:0.0:1789762870.088507:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343380467616 time 343330467616 state 0x1 ref 1 00000100:00100000:3.0:1789762870.133467:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343380467616 time 343380535672 state 0x0 ref 1 00000100:00100000:3.0:1789762870.133486:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067452544:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762870.133527:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766fb lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.133541:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.133544:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.133549:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766fb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.133559:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766fb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.136040:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766fb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.136052:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766fb lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.136059:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766fb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.136065:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f63ad400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762870.136068:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766fb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.136074:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766fb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.136088:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766fb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.136100:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e95e0298/0xdf5a7ac59f1766fb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762870.136151:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067452544:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2671us (47754us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762870.136157:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 650 00000100:00100000:0.0:1789762870.136165:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343380467616 time 343380535672 state 0x2 ref 0 00000100:00100000:2.0:1789762870.138389:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762870.138434:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067452672 00000100:00100000:1.0:1789762870.138442:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067452672 00000100:00100000:1.0:1789762870.138477:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343380467616 time 343380535672 state 0x0 ref 1 00000100:00100000:1.0:1789762870.138487:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343430535672 time 343380535672 state 0x1 ref 1 00000100:00100000:1.0:1789762870.183500:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343430535672 time 343430570816 state 0x0 ref 1 00000100:00100000:1.0:1789762870.183515:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067452672:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762870.183540:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176702 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.183552:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762870.183555:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762870.183559:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176702 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.183572:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176702 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.185937:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176702 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.185949:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176702 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.185959:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176702 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.185968:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762870.185972:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176702 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.185981:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176702 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.185997:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176702 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.186006:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176702 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762870.186061:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067452672:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2549us (47678us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762870.186070:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 651 00000100:00100000:1.0:1789762870.186079:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343430535672 time 343430570816 state 0x2 ref 0 00000100:00100000:2.0:1789762870.187909:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762870.187955:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067452800 00000100:00100000:3.0:1789762870.187963:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067452800 00000100:00100000:3.0:1789762870.187991:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343430535672 time 343430570816 state 0x0 ref 1 00000100:00100000:3.0:1789762870.188000:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343480570816 time 343430570816 state 0x1 ref 1 00000100:00100000:3.0:1789762870.233564:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343480570816 time 343480624259 state 0x0 ref 1 00000100:00100000:3.0:1789762870.233590:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067452800:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762870.233625:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176709 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.233638:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.233642:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.233646:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176709 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.233668:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176709 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.236415:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176709 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.236427:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176709 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.236437:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176709 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.236445:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762870.236449:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176709 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.236458:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176709 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.236476:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176709 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.236486:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176709 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762870.236550:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067452800:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2967us (48641us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762870.236560:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 652 00000100:00100000:3.0:1789762870.236569:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343480570816 time 343480624259 state 0x2 ref 0 00000100:00100000:3.0:1789762870.238441:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762870.238520:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067452928 00000100:00100000:0.0:1789762870.238529:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067452928 00000100:00100000:0.0:1789762870.238564:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343480570816 time 343480624259 state 0x0 ref 1 00000100:00100000:0.0:1789762870.238574:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343530624259 time 343480624259 state 0x1 ref 1 00000100:00100000:3.0:1789762870.283651:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343530624259 time 343530715186 state 0x0 ref 1 00000100:00100000:3.0:1789762870.283672:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067452928:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762870.283707:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176710 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.283721:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.283724:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.283729:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176710 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.283740:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176710 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.286482:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176710 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.286493:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176710 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.286508:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176710 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.286514:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762870.286517:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176710 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.286525:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176710 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.286542:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176710 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.286551:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f176710 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762870.286606:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067452928:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2940us (48165us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762870.286613:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 653 00000100:00100000:3.0:1789762870.286621:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343530624259 time 343530715186 state 0x2 ref 0 00000100:00100000:3.0:1789762870.288393:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762870.288442:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067453056 00000100:00100000:1.0:1789762870.288451:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067453056 00000100:00100000:1.0:1789762870.288491:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343530624259 time 343530715186 state 0x0 ref 1 00000100:00100000:1.0:1789762870.288501:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343580715186 time 343530715186 state 0x1 ref 1 00000100:00100000:0.0:1789762870.333738:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343580715186 time 343580799100 state 0x0 ref 1 00000100:00100000:0.0:1789762870.333757:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067453056:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762870.333789:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d4779d3/0xdf5a7ac59f176717 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.333807:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762870.333810:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762870.333815:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d4779d3/0xdf5a7ac59f176717 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.333842:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d4779d3/0xdf5a7ac59f176717 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.336424:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d4779d3/0xdf5a7ac59f176717 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.336433:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d4779d3/0xdf5a7ac59f176717 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.336439:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d4779d3/0xdf5a7ac59f176717 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.336444:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795ae00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762870.336447:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d4779d3/0xdf5a7ac59f176717 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.336454:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d4779d3/0xdf5a7ac59f176717 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.336467:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d4779d3/0xdf5a7ac59f176717 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.336473:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d4779d3/0xdf5a7ac59f176717 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762870.336528:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067453056:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2776us (48134us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762870.336538:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 654 00000100:00100000:0.0:1789762870.336549:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343580715186 time 343580799100 state 0x2 ref 0 00000100:00100000:2.0:1789762870.338420:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762870.338459:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067453184 00000100:00100000:3.0:1789762870.338467:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067453184 00000100:00100000:3.0:1789762870.338501:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343580715186 time 343580799100 state 0x0 ref 1 00000100:00100000:3.0:1789762870.338517:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343630799100 time 343580799100 state 0x1 ref 1 00000100:00100000:1.0:1789762870.383846:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343630799100 time 343630906881 state 0x0 ref 1 00000100:00100000:1.0:1789762870.383866:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9255500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067453184:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762870.383895:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17671e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.383907:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762870.383911:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762870.383941:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17671e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.383952:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17671e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.386489:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17671e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.386497:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17671e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.386507:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17671e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.386513:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762870.386516:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17671e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.386522:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17671e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.386534:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17671e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.386539:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17671e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762870.386580:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9255500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067453184:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2720us (48161us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762870.386587:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 655 00000100:00100000:1.0:1789762870.386596:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343630799100 time 343630906881 state 0x2 ref 0 00000100:00100000:2.0:1789762870.388446:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762870.388493:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067453312 00000100:00100000:3.0:1789762870.388501:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067453312 00000100:00100000:3.0:1789762870.388538:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343630799100 time 343630906881 state 0x0 ref 1 00000100:00100000:3.0:1789762870.388553:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343680906881 time 343630906881 state 0x1 ref 1 00000100:00100000:3.0:1789762870.433912:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343680906881 time 343680974582 state 0x0 ref 1 00000100:00100000:3.0:1789762870.433966:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067453312:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762870.434007:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f176725 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.434025:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.434029:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.434035:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f176725 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.434049:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f176725 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.436650:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f176725 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.436669:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f176725 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.436676:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f176725 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.436681:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762870.436684:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f176725 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.436691:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f176725 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.436705:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f176725 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.436710:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f176725 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762870.436756:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067453312:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2799us (48311us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762870.436763:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 656 00000100:00100000:3.0:1789762870.436770:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343680906881 time 343680974582 state 0x2 ref 0 00000100:00100000:3.0:1789762870.438592:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762870.438672:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067453440 00000100:00100000:0.0:1789762870.438681:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067453440 00000100:00100000:0.0:1789762870.438717:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343680906881 time 343680974582 state 0x0 ref 1 00000100:00100000:0.0:1789762870.438728:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343730974582 time 343680974582 state 0x1 ref 1 00000100:00100000:0.0:1789762870.483961:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343730974582 time 343731024248 state 0x0 ref 1 00000100:00100000:0.0:1789762870.483989:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067453440:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762870.484023:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d4f4e5e3/0xdf5a7ac59f17672c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.484038:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762870.484043:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762870.484050:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4f4e5e3/0xdf5a7ac59f17672c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.484065:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4f4e5e3/0xdf5a7ac59f17672c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.486756:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d4f4e5e3/0xdf5a7ac59f17672c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.486768:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d4f4e5e3/0xdf5a7ac59f17672c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.486775:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d4f4e5e3/0xdf5a7ac59f17672c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.486780:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e3b8c800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762870.486783:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d4f4e5e3/0xdf5a7ac59f17672c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.486789:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d4f4e5e3/0xdf5a7ac59f17672c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.486802:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d4f4e5e3/0xdf5a7ac59f17672c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.486807:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4f4e5e3/0xdf5a7ac59f17672c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762870.486853:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067453440:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2875us (48262us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762870.486860:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 657 00000100:00100000:3.0:1789762870.486868:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343730974582 time 343731024248 state 0x2 ref 0 00000100:00100000:3.0:1789762870.488966:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762870.489038:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067453568 00000100:00100000:1.0:1789762870.489047:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067453568 00000100:00100000:1.0:1789762870.489089:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343730974582 time 343731024248 state 0x0 ref 1 00000100:00100000:1.0:1789762870.489100:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343781024248 time 343731024248 state 0x1 ref 1 00000100:00100000:1.0:1789762870.534043:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343781024248 time 343781097135 state 0x0 ref 1 00000100:00100000:1.0:1789762870.534065:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067453568:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762870.534110:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176733 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.534124:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762870.534128:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762870.534132:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176733 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.534144:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176733 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.536982:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176733 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.536993:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176733 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.537001:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176733 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.537007:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762870.537012:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176733 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.537018:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176733 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.537031:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176733 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.537037:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176733 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762870.537084:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067453568:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3027us (48119us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762870.537091:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 658 00000100:00100000:1.0:1789762870.537099:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343781024248 time 343781097135 state 0x2 ref 0 00000100:00100000:3.0:1789762870.539153:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762870.539195:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067453696 00000100:00100000:0.0:1789762870.539204:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067453696 00000100:00100000:0.0:1789762870.539238:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343781024248 time 343781097135 state 0x0 ref 1 00000100:00100000:0.0:1789762870.539249:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343831097135 time 343781097135 state 0x1 ref 1 00000100:00100000:3.0:1789762870.584097:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343831097135 time 343831166521 state 0x0 ref 1 00000100:00100000:3.0:1789762870.584127:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067453696:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762870.584156:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17673a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.584169:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.584172:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.584177:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17673a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.584187:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17673a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.587042:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17673a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.587052:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17673a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.587058:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17673a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.587063:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762870.587067:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17673a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.587073:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17673a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.587095:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17673a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.587110:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1bdba8/0xdf5a7ac59f17673a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762870.587165:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067453696:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3043us (48012us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762870.587171:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 659 00000100:00100000:0.0:1789762870.587178:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343831097135 time 343831166521 state 0x2 ref 0 00000100:00100000:0.0:1789762870.589029:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762870.589074:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067453824 00000100:00100000:3.0:1789762870.589081:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067453824 00000100:00100000:3.0:1789762870.589114:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343831097135 time 343831166521 state 0x0 ref 1 00000100:00100000:3.0:1789762870.589124:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343881166521 time 343831166521 state 0x1 ref 1 00000100:00100000:3.0:1789762870.634172:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343881166521 time 343881235420 state 0x0 ref 1 00000100:00100000:3.0:1789762870.634192:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067453824:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762870.634239:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176741 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.634253:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.634257:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.634264:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176741 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.634279:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176741 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.637151:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176741 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.637165:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176741 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.637174:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176741 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.637183:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762870.637187:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176741 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.637196:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176741 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.637228:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176741 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.637237:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176741 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762870.637294:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067453824:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3106us (48263us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762870.637305:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 660 00000100:00100000:3.0:1789762870.637329:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343881166521 time 343881235420 state 0x2 ref 0 00000100:00100000:2.0:1789762870.639236:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762870.639298:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067453952 00000100:00100000:1.0:1789762870.639305:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067453952 00000100:00100000:1.0:1789762870.639358:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343881166521 time 343881235420 state 0x0 ref 1 00000100:00100000:1.0:1789762870.639372:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343931235420 time 343881235420 state 0x1 ref 1 00000100:00100000:3.0:1789762870.684258:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343931235420 time 343931325506 state 0x0 ref 1 00000100:00100000:3.0:1789762870.684279:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9256680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067453952:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762870.684310:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176748 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.684337:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.684340:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.684345:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176748 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.684356:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176748 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.687943:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176748 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.687954:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176748 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.687961:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176748 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.687966:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762870.687969:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176748 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.687975:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176748 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.687986:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176748 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.687991:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176748 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762870.688033:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9256680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067453952:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3760us (48798us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762870.688040:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 661 00000100:00100000:3.0:1789762870.688053:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343931235420 time 343931325506 state 0x2 ref 0 00000100:00100000:2.0:1789762870.690019:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762870.690062:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067454080 00000100:00100000:0.0:1789762870.690070:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067454080 00000100:00100000:0.0:1789762870.690100:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343931235420 time 343931325506 state 0x0 ref 1 00000100:00100000:0.0:1789762870.690110:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343981325506 time 343931325506 state 0x1 ref 1 00000100:00100000:1.0:1789762870.734374:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343981325506 time 343981444132 state 0x0 ref 1 00000100:00100000:1.0:1789762870.734392:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9257480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067454080:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762870.734417:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17674f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.734429:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762870.734432:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762870.734436:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17674f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.734445:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17674f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.737666:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17674f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.737676:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17674f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.737683:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17674f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.737693:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762870.737696:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17674f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.737703:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17674f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.737716:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17674f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.737722:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17674f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762870.737780:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9257480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067454080:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3392us (47761us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762870.737789:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 662 00000100:00100000:1.0:1789762870.737798:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343981325506 time 343981444132 state 0x2 ref 0 00000100:00100000:2.0:1789762870.739842:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762870.739906:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067454208 00000100:00100000:3.0:1789762870.739944:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067454208 00000100:00100000:3.0:1789762870.740003:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 343981325506 time 343981444132 state 0x0 ref 1 00000100:00100000:3.0:1789762870.740018:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344031444132 time 343981444132 state 0x1 ref 1 00000100:00100000:0.0:1789762870.784449:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344031444132 time 344031519293 state 0x0 ref 1 00000100:00100000:0.0:1789762870.784467:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9256a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067454208:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762870.784496:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f176756 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.784518:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762870.784521:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762870.784526:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f176756 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.784548:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f176756 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.787547:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f176756 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.787557:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f176756 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.787563:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f176756 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.787568:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762870.787573:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f176756 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.787580:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f176756 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.787594:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f176756 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.787599:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f176756 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762870.787653:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9256a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067454208:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3191us (47812us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762870.787660:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 663 00000100:00100000:0.0:1789762870.787667:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344031444132 time 344031519293 state 0x2 ref 0 00000100:00100000:0.0:1789762870.789659:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762870.789743:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067454336 00000100:00100000:3.0:1789762870.789757:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067454336 00000100:00100000:3.0:1789762870.789834:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344031444132 time 344031519293 state 0x0 ref 1 00000100:00100000:3.0:1789762870.789851:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344081519293 time 344031519293 state 0x1 ref 1 00000100:00100000:3.0:1789762870.834561:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344081519293 time 344081603567 state 0x0 ref 1 00000100:00100000:3.0:1789762870.834598:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067454336:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762870.834674:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f17675d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.834701:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.834709:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.834718:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f17675d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.834739:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f17675d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.838766:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f17675d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.838784:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f17675d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.838796:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f17675d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.838821:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762870.838827:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f17675d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.838834:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f17675d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.838855:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f17675d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.838862:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f17675d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762870.838963:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067454336:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4379us (49303us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762870.838975:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 664 00000100:00100000:3.0:1789762870.838994:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344081519293 time 344081603567 state 0x2 ref 0 00000100:00100000:2.0:1789762870.841036:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762870.841123:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067454464 00000100:00100000:1.0:1789762870.841135:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067454464 00000100:00100000:1.0:1789762870.841187:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344081519293 time 344081603567 state 0x0 ref 1 00000100:00100000:1.0:1789762870.841209:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344131603567 time 344081603567 state 0x1 ref 1 00000100:00100000:1.0:1789762870.884603:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344131603567 time 344131670903 state 0x0 ref 1 00000100:00100000:1.0:1789762870.884627:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9257100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067454464:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762870.884660:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176764 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.884674:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762870.884678:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762870.884683:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176764 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.884693:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176764 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.887377:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176764 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.887389:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176764 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.887396:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176764 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.887401:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762870.887404:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176764 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.887411:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176764 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.887425:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176764 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762870.887431:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176764 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762870.887480:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9257100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067454464:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2862us (46451us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762870.887487:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 665 00000100:00100000:1.0:1789762870.887495:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344131603567 time 344131670903 state 0x2 ref 0 00000100:00100000:0.0:1789762870.889385:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762870.889445:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067454592 00000100:00100000:3.0:1789762870.889459:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067454592 00000100:00100000:3.0:1789762870.889514:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344131603567 time 344131670903 state 0x0 ref 1 00000100:00100000:3.0:1789762870.889527:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344181670903 time 344131670903 state 0x1 ref 1 00010000:00010000:3.0:1789762870.912080:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762870.934743:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344181670903 time 344181807956 state 0x0 ref 1 00000100:00100000:0.0:1789762870.934768:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067454592:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762870.934804:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17676b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.934818:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762870.934821:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762870.934826:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17676b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762870.934836:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17676b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.937822:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17676b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.937840:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17676b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.937848:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17676b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.937854:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762870.937860:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17676b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.937868:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17676b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.937890:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17676b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.937940:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f17676b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762870.938030:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067454592:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3269us (48642us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762870.938047:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 666 00000100:00100000:3.0:1789762870.938058:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344181670903 time 344181807956 state 0x2 ref 0 00000100:00100000:2.0:1789762870.939881:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762870.939945:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067454720 00000100:00100000:3.0:1789762870.939955:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067454720 00000100:00100000:3.0:1789762870.940011:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344181670903 time 344181807956 state 0x0 ref 1 00000100:00100000:3.0:1789762870.940025:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344231807956 time 344181807956 state 0x1 ref 1 00000100:00100000:3.0:1789762870.984798:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344231807956 time 344231860476 state 0x0 ref 1 00000100:00100000:3.0:1789762870.984823:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067454720:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762870.984873:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176772 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.984892:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.984896:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762870.984902:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176772 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.984952:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176772 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.987844:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176772 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.987855:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176772 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.987862:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176772 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.987867:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762870.987871:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176772 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.987877:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176772 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.987892:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176772 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762870.987897:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176772 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762870.987996:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067454720:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3180us (48116us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762870.988008:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 667 00000100:00100000:3.0:1789762870.988018:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344231807956 time 344231860476 state 0x2 ref 0 00000100:00100000:2.0:1789762870.989852:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762870.989899:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067454848 00000100:00100000:1.0:1789762870.989908:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067454848 00000100:00100000:1.0:1789762870.989985:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344231807956 time 344231860476 state 0x0 ref 1 00000100:00100000:1.0:1789762870.989997:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344281860476 time 344231860476 state 0x1 ref 1 00000100:00100000:0.0:1789762871.034893:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344281860476 time 344281959184 state 0x0 ref 1 00000100:00100000:0.0:1789762871.034945:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9254e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067454848:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:3.0:1789762871.034957:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762871.034999:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000003ebf257/0xdf5a7ac59f176779 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.035018:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762871.035022:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762871.035027:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003ebf257/0xdf5a7ac59f176779 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.035038:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003ebf257/0xdf5a7ac59f176779 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.038219:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000003ebf257/0xdf5a7ac59f176779 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.038235:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003ebf257/0xdf5a7ac59f176779 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.038244:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003ebf257/0xdf5a7ac59f176779 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.038251:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e3b97600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762871.038256:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003ebf257/0xdf5a7ac59f176779 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.038264:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003ebf257/0xdf5a7ac59f176779 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.038284:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003ebf257/0xdf5a7ac59f176779 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.038307:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003ebf257/0xdf5a7ac59f176779 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762871.038407:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9254e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067454848:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3490us (48553us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762871.038416:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 668 00000100:00100000:3.0:1789762871.038427:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344281860476 time 344281959184 state 0x2 ref 0 00000100:00100000:2.0:1789762871.040339:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762871.040377:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067454976 00000100:00100000:3.0:1789762871.040387:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067454976 00000100:00100000:3.0:1789762871.040433:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344281860476 time 344281959184 state 0x0 ref 1 00000100:00100000:3.0:1789762871.040445:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344331959184 time 344281959184 state 0x1 ref 1 00000100:00100000:1.0:1789762871.085008:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344331959184 time 344332076677 state 0x0 ref 1 00000100:00100000:1.0:1789762871.085030:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9255c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067454976:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762871.085058:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176780 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.085071:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762871.085076:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762871.085080:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176780 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.085090:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176780 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.088435:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176780 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.088448:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176780 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.088455:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176780 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.088460:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762871.088463:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176780 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.088470:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176780 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.088483:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176780 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.088488:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176780 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762871.088541:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9255c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067454976:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3518us (48202us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762871.088549:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 669 00000100:00100000:1.0:1789762871.088558:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344331959184 time 344332076677 state 0x2 ref 0 00000100:00100000:3.0:1789762871.090421:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762871.090473:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067455104 00000100:00100000:1.0:1789762871.090482:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067455104 00000100:00100000:1.0:1789762871.090551:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344331959184 time 344332076677 state 0x0 ref 1 00000100:00100000:1.0:1789762871.090568:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344382076677 time 344332076677 state 0x1 ref 1 00000100:00100000:3.0:1789762871.135146:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344382076677 time 344382211526 state 0x0 ref 1 00000100:00100000:3.0:1789762871.135167:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067455104:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762871.135204:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176787 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.135218:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.135221:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.135226:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176787 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.135250:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176787 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.138014:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176787 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.138026:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176787 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.138033:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176787 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.138039:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762871.138042:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176787 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.138049:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176787 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.138065:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176787 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.138071:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad61a1af/0xdf5a7ac59f176787 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762871.138118:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067455104:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2957us (47700us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762871.138125:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 670 00000100:00100000:1.0:1789762871.138132:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344382076677 time 344382211526 state 0x2 ref 0 00000100:00100000:3.0:1789762871.139967:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762871.140015:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067455232 00000100:00100000:0.0:1789762871.140022:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067455232 00000100:00100000:0.0:1789762871.140057:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344382076677 time 344382211526 state 0x0 ref 1 00000100:00100000:0.0:1789762871.140067:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344432211526 time 344382211526 state 0x1 ref 1 00000100:00100000:0.0:1789762871.185211:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344432211526 time 344432277072 state 0x0 ref 1 00000100:00100000:0.0:1789762871.185232:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067455232:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762871.185266:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f17678e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.185280:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762871.185283:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762871.185289:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f17678e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.185329:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f17678e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.188012:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f17678e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.188025:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f17678e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.188035:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f17678e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.188043:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37fddee000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762871.188048:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f17678e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.188057:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f17678e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.188074:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f17678e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.188083:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f17678e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762871.188150:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067455232:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2924us (48182us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762871.188160:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 671 00000100:00100000:0.0:1789762871.188171:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344432211526 time 344432277072 state 0x2 ref 0 00000100:00100000:3.0:1789762871.189952:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762871.190024:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067455360 00000100:00100000:1.0:1789762871.190033:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067455360 00000100:00100000:1.0:1789762871.190072:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344432211526 time 344432277072 state 0x0 ref 1 00000100:00100000:1.0:1789762871.190082:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344482277072 time 344432277072 state 0x1 ref 1 00000100:00100000:3.0:1789762871.235328:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344482277072 time 344482381391 state 0x0 ref 1 00000100:00100000:3.0:1789762871.235352:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067455360:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762871.235389:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176795 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.235403:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.235407:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.235411:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176795 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.235422:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176795 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.238195:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176795 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.238206:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176795 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.238213:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176795 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.238224:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762871.238228:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176795 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.238234:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176795 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.238248:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176795 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.238254:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000552eb460/0xdf5a7ac59f176795 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762871.238300:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067455360:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2956us (48349us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762871.238308:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 672 00000100:00100000:1.0:1789762871.238329:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344482277072 time 344482381391 state 0x2 ref 0 00000100:00100000:3.0:1789762871.240190:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762871.240225:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067455488 00000100:00100000:1.0:1789762871.240233:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067455488 00000100:00100000:1.0:1789762871.240267:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344482277072 time 344482381391 state 0x0 ref 1 00000100:00100000:1.0:1789762871.240277:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344532381391 time 344482381391 state 0x1 ref 1 00000100:00100000:3.0:1789762871.285426:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344532381391 time 344532490975 state 0x0 ref 1 00000100:00100000:3.0:1789762871.285447:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067455488:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762871.285483:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17679c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.285511:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.285514:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.285520:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17679c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.285531:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17679c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.288237:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17679c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.288249:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17679c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.288255:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17679c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.288260:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273d000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762871.288264:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17679c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.288271:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17679c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.288284:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17679c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.288289:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cda16c88/0xdf5a7ac59f17679c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762871.288363:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067455488:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2921us (48175us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762871.288372:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 673 00000100:00100000:0.0:1789762871.288381:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344532381391 time 344532490975 state 0x2 ref 0 00000100:00100000:2.0:1789762871.290267:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762871.290341:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067455616 00000100:00100000:0.0:1789762871.290349:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067455616 00000100:00100000:0.0:1789762871.290386:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344532381391 time 344532490975 state 0x0 ref 1 00000100:00100000:0.0:1789762871.290396:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344582490975 time 344532490975 state 0x1 ref 1 00000100:00100000:3.0:1789762871.335513:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344582490975 time 344582579239 state 0x0 ref 1 00000100:00100000:3.0:1789762871.335534:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067455616:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762871.335568:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767a3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.335582:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.335585:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.335590:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767a3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.335600:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767a3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.338473:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767a3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.338511:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767a3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.338521:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767a3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.338530:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762871.338535:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767a3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.338546:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767a3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.338566:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767a3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.338576:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767a3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762871.338642:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067455616:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3112us (48375us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762871.338655:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 674 00000100:00100000:3.0:1789762871.338667:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344582490975 time 344582579239 state 0x2 ref 0 00000100:00100000:2.0:1789762871.340845:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762871.340894:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067455744 00000100:00100000:1.0:1789762871.340904:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067455744 00000100:00100000:1.0:1789762871.340966:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344582490975 time 344582579239 state 0x0 ref 1 00000100:00100000:1.0:1789762871.340977:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344632579239 time 344582579239 state 0x1 ref 1 00000100:00100000:0.0:1789762871.385625:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344632579239 time 344632681618 state 0x0 ref 1 00000100:00100000:0.0:1789762871.385654:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9254000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067455744:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762871.385695:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f1767aa lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.385713:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762871.385718:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762871.385724:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f1767aa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.385740:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f1767aa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.388682:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f1767aa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.388709:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f1767aa lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.388717:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f1767aa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.388722:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37fddee000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762871.388726:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f1767aa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.388734:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f1767aa lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.388751:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f1767aa lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.388756:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ca087b3/0xdf5a7ac59f1767aa lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762871.388832:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9254000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067455744:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3187us (47985us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762871.388843:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 675 00000100:00100000:3.0:1789762871.388853:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344632579239 time 344632681618 state 0x2 ref 0 00000100:00100000:2.0:1789762871.390748:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762871.390793:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067455872 00000100:00100000:0.0:1789762871.390801:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067455872 00000100:00100000:0.0:1789762871.390840:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344632579239 time 344632681618 state 0x0 ref 1 00000100:00100000:0.0:1789762871.390850:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344682681618 time 344632681618 state 0x1 ref 1 00000100:00100000:1.0:1789762871.435732:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344682681618 time 344682792564 state 0x0 ref 1 00000100:00100000:1.0:1789762871.435757:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9256300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067455872:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762871.435789:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.435804:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762871.435807:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762871.435812:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.435821:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.438563:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.438574:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.438581:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.438587:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762871.438591:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.438597:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.438611:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.438616:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762871.438666:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9256300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067455872:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2919us (47919us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762871.438672:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 676 00000100:00100000:1.0:1789762871.438679:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344682681618 time 344682792564 state 0x2 ref 0 00000100:00100000:3.0:1789762871.440554:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762871.440638:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067456000 00000100:00100000:1.0:1789762871.440646:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067456000 00000100:00100000:1.0:1789762871.440682:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344682681618 time 344682792564 state 0x0 ref 1 00000100:00100000:1.0:1789762871.440692:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344732792564 time 344682792564 state 0x1 ref 1 00000100:00100000:1.0:1789762871.485798:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344732792564 time 344732857658 state 0x0 ref 1 00000100:00100000:1.0:1789762871.485824:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067456000:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762871.485857:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.485871:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762871.485874:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762871.485878:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.485889:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.488525:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.488534:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.488540:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.488545:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762871.488548:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.488554:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.488567:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.488572:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767b8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762871.488620:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067456000:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2807us (48071us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762871.488633:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 677 00000100:00100000:1.0:1789762871.488640:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344732792564 time 344732857658 state 0x2 ref 0 00000100:00100000:3.0:1789762871.490486:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762871.490559:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067456128 00000100:00100000:0.0:1789762871.490570:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067456128 00000100:00100000:0.0:1789762871.490634:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344732792564 time 344732857658 state 0x0 ref 1 00000100:00100000:0.0:1789762871.490647:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344782857658 time 344732857658 state 0x1 ref 1 00000100:00100000:3.0:1789762871.535862:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344782857658 time 344782928981 state 0x0 ref 1 00000100:00100000:3.0:1789762871.535884:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067456128:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762871.535948:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767bf lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.535969:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.535974:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.535982:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767bf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.536000:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767bf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.538837:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767bf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.538851:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767bf lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.538860:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767bf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.538866:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762871.538869:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767bf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.538888:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767bf lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.538909:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767bf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.538915:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000957e99fc/0xdf5a7ac59f1767bf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762871.538990:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067456128:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3113us (48509us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762871.538997:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 678 00000100:00100000:0.0:1789762871.539006:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344782857658 time 344782928981 state 0x2 ref 0 00000100:00100000:2.0:1789762871.541004:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762871.541046:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067456256 00000100:00100000:1.0:1789762871.541057:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067456256 00000100:00100000:1.0:1789762871.541115:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344782857658 time 344782928981 state 0x0 ref 1 00000100:00100000:1.0:1789762871.541128:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344832928981 time 344782928981 state 0x1 ref 1 00000100:00100000:3.0:1789762871.585965:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344832928981 time 344833029448 state 0x0 ref 1 00000100:00100000:3.0:1789762871.585986:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9257b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067456256:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762871.586026:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767c6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.586041:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.586044:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.586050:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767c6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.586076:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767c6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.589097:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767c6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.589108:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767c6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.589114:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767c6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.589119:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762871.589123:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767c6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.589129:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767c6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.589142:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767c6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.589147:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767c6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762871.589213:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9257b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067456256:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3234us (48215us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762871.589223:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 679 00000100:00100000:3.0:1789762871.589232:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344832928981 time 344833029448 state 0x2 ref 0 00000100:00100000:0.0:1789762871.591189:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762871.591271:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067456384 00000100:00100000:1.0:1789762871.591281:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067456384 00000100:00100000:1.0:1789762871.591345:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344832928981 time 344833029448 state 0x0 ref 1 00000100:00100000:1.0:1789762871.591356:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344883029448 time 344833029448 state 0x1 ref 1 00000100:00100000:3.0:1789762871.636051:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344883029448 time 344883115844 state 0x0 ref 1 00000100:00100000:3.0:1789762871.636073:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067456384:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762871.636109:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767cd lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.636122:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.636125:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.636131:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767cd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.636141:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767cd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.639181:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767cd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.639193:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767cd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.639215:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767cd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.639223:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762871.639228:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767cd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.639236:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767cd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.639253:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767cd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.639260:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767cd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762871.639324:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067456384:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3257us (48135us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762871.639332:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 680 00000100:00100000:3.0:1789762871.639340:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344883029448 time 344883115844 state 0x2 ref 0 00000100:00100000:2.0:1789762871.641123:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762871.641183:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067456512 00000100:00100000:0.0:1789762871.641192:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067456512 00000100:00100000:0.0:1789762871.641239:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344883029448 time 344883115844 state 0x0 ref 1 00000100:00100000:0.0:1789762871.641251:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344933115844 time 344883115844 state 0x1 ref 1 00000100:00100000:3.0:1789762871.686183:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344933115844 time 344933242460 state 0x0 ref 1 00000100:00100000:3.0:1789762871.686208:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067456512:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762871.686272:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767d4 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.686292:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.686298:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.686318:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767d4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.686336:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767d4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.689445:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767d4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.689458:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767d4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.689466:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767d4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.689472:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d273c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762871.689475:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767d4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.689482:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767d4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.689498:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767d4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.689503:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006066b40e/0xdf5a7ac59f1767d4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762871.689559:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067456512:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3359us (48443us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762871.689567:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 681 00000100:00100000:1.0:1789762871.689575:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344933115844 time 344933242460 state 0x2 ref 0 00000100:00100000:2.0:1789762871.691453:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762871.691500:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067456640 00000100:00100000:3.0:1789762871.691509:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067456640 00000100:00100000:3.0:1789762871.691551:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344933115844 time 344933242460 state 0x0 ref 1 00000100:00100000:3.0:1789762871.691563:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344983242460 time 344933242460 state 0x1 ref 1 00000100:00100000:0.0:1789762871.736337:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344983242460 time 344983370284 state 0x0 ref 1 00000100:00100000:0.0:1789762871.736393:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067456640:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762871.736497:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f1767db lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.736521:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762871.736527:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762871.736535:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f1767db lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.736555:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f1767db lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.740787:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f1767db lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.740809:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f1767db lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.740826:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f1767db lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.740841:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f1a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762871.740851:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f1767db lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.740870:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f1767db lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.740970:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f1767db lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.740989:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a58bdb0/0xdf5a7ac59f1767db lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762871.741077:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067456640:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4694us (49621us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762871.741095:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 682 00000100:00100000:0.0:1789762871.741113:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344983242460 time 344983370284 state 0x2 ref 0 00000100:00100000:0.0:1789762871.743684:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762871.743756:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067456768 00000100:00100000:3.0:1789762871.743769:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067456768 00000100:00100000:3.0:1789762871.743831:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 344983242460 time 344983370284 state 0x0 ref 1 00000100:00100000:3.0:1789762871.743849:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345033370284 time 344983370284 state 0x1 ref 1 00000100:00100000:1.0:1789762871.786453:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345033370284 time 345033512133 state 0x0 ref 1 00000100:00100000:1.0:1789762871.786486:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067456768:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762871.786536:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.786556:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762871.786561:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762871.786568:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.786585:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.790621:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.790637:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.790646:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.790653:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762871.790658:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.790667:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.790687:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.790695:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762871.790768:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067456768:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4294us (47086us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762871.790778:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 683 00000100:00100000:1.0:1789762871.790789:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345033370284 time 345033512133 state 0x2 ref 0 00000100:00100000:3.0:1789762871.793409:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762871.793458:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067456896 00000100:00100000:1.0:1789762871.793470:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067456896 00000100:00100000:1.0:1789762871.793520:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345033370284 time 345033512133 state 0x0 ref 1 00000100:00100000:1.0:1789762871.793535:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345083512133 time 345033512133 state 0x1 ref 1 00000100:00100000:1.0:1789762871.808163:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074703360:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762871.808231:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762871.808266:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074703360 00000100:00100000:0.0:1789762871.808276:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074703360 00000100:00100000:0.0:1789762871.808356:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 335 00000100:00100000:0.0:1789762871.808375:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074703360:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762871.808425:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762871.808496:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074703360:12345-0@lo:41:osp-pre-1-0.0 Request processed in 128us (264us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762871.808506:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 335 00000100:00100000:1.0:1789762871.808569:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074703360:0@lo:41:osp-pre-1-0.0 00000001:00080000:0.0:1789762871.823277:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967399 is committed 00000100:00100000:1.0:1789762871.836528:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345083512133 time 345083587574 state 0x0 ref 1 00000100:00100000:1.0:1789762871.836557:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067456896:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762871.836610:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e9 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.836633:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762871.836639:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762871.836648:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.836667:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.840252:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.840272:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.840303:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.840333:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762871.840340:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.840371:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.840402:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.840414:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f1767e9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762871.840485:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067456896:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3934us (47077us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762871.840498:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 684 00000100:00100000:1.0:1789762871.840528:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345083512133 time 345083587574 state 0x2 ref 0 00000100:00100000:3.0:1789762871.843274:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762871.843347:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067457024 00000100:00100000:0.0:1789762871.843366:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067457024 00000100:00100000:0.0:1789762871.843428:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345083512133 time 345083587574 state 0x0 ref 1 00000100:00100000:0.0:1789762871.843461:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345133587574 time 345083587574 state 0x1 ref 1 00000100:00100000:0.0:1789762871.886585:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345133587574 time 345133647789 state 0x0 ref 1 00000100:00100000:0.0:1789762871.886608:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067457024:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762871.886649:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1767f0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.886665:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762871.886669:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762871.886674:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1767f0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.886687:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1767f0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.889670:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1767f0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.889696:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1767f0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.889704:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1767f0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.889709:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762871.889713:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1767f0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.889720:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1767f0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.889736:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1767f0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.889740:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1767f0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762871.889793:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067457024:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3192us (46523us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762871.889801:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 685 00000100:00100000:3.0:1789762871.889810:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345133587574 time 345133647789 state 0x2 ref 0 00000100:00100000:3.0:1789762871.891733:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762871.891791:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067457152 00000100:00100000:1.0:1789762871.891801:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067457152 00000100:00100000:1.0:1789762871.891854:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345133587574 time 345133647789 state 0x0 ref 1 00000100:00100000:1.0:1789762871.891867:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345183647789 time 345133647789 state 0x1 ref 1 00010000:00010000:1.0:1789762871.936063:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762871.936650:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345183647789 time 345183717970 state 0x0 ref 1 00000100:00100000:3.0:1789762871.936671:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067457152:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762871.936710:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767f7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.936726:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.936730:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.936735:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767f7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.936745:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767f7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.939636:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767f7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.939648:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767f7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.939656:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767f7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.939662:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762871.939666:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767f7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.939683:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767f7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.939702:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767f7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.939707:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767f7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762871.939759:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067457152:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3093us (48026us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762871.939767:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 686 00000100:00100000:3.0:1789762871.939775:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345183647789 time 345183717970 state 0x2 ref 0 00000100:00100000:3.0:1789762871.941630:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762871.941683:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067457280 00000100:00100000:1.0:1789762871.941691:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067457280 00000100:00100000:1.0:1789762871.941736:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345183647789 time 345183717970 state 0x0 ref 1 00000100:00100000:1.0:1789762871.941746:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345233717970 time 345183717970 state 0x1 ref 1 00000100:00100000:3.0:1789762871.986801:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345233717970 time 345233855249 state 0x0 ref 1 00000100:00100000:3.0:1789762871.986838:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067457280:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762871.986896:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767fe lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.986971:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.986981:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762871.986994:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767fe lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.987086:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767fe lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.991574:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767fe lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.991594:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767fe lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.991606:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767fe lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.991615:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762871.991621:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767fe lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.991650:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767fe lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.991671:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767fe lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.991680:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1767fe lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762871.991777:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067457280:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4950us (50144us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762871.991789:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 687 00000100:00100000:3.0:1789762871.991802:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345233717970 time 345233855249 state 0x2 ref 0 00000100:00100000:3.0:1789762871.994269:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762871.994426:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067457408 00000100:00100000:0.0:1789762871.994469:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067457408 00000100:00100000:0.0:1789762871.994541:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345233717970 time 345233855249 state 0x0 ref 1 00000100:00100000:0.0:1789762871.994560:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345283855249 time 345233855249 state 0x1 ref 1 00000100:00100000:3.0:1789762872.037017:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345283855249 time 345284053945 state 0x0 ref 1 00000100:00100000:3.0:1789762872.037055:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067457408:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762872.037115:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176805 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.037139:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.037145:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.037152:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176805 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.037171:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176805 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.042078:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176805 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.042108:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176805 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.042123:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176805 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.042140:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762872.042147:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176805 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.042157:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176805 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.042180:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176805 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.042187:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176805 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762872.042282:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067457408:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5231us (48008us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762872.042295:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 688 00000100:00100000:1.0:1789762872.042331:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345283855249 time 345284053945 state 0x2 ref 0 00000100:00100000:2.0:1789762872.045047:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762872.045103:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067457536 00000100:00100000:3.0:1789762872.045115:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067457536 00000100:00100000:3.0:1789762872.045179:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345283855249 time 345284053945 state 0x0 ref 1 00000100:00100000:3.0:1789762872.045239:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345334053945 time 345284053945 state 0x1 ref 1 00000100:00100000:0.0:1789762872.087090:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345334053945 time 345334153179 state 0x0 ref 1 00000100:00100000:0.0:1789762872.087129:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067457536:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762872.087173:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000eef54246/0xdf5a7ac59f17680c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.087191:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762872.087196:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762872.087203:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eef54246/0xdf5a7ac59f17680c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.087235:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eef54246/0xdf5a7ac59f17680c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.091663:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000eef54246/0xdf5a7ac59f17680c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.091682:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eef54246/0xdf5a7ac59f17680c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.091694:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eef54246/0xdf5a7ac59f17680c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.091703:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c8e5d600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762872.091709:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eef54246/0xdf5a7ac59f17680c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.091737:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eef54246/0xdf5a7ac59f17680c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.091759:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eef54246/0xdf5a7ac59f17680c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.091768:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eef54246/0xdf5a7ac59f17680c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762872.091844:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067457536:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4719us (46798us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762872.091855:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 689 00000100:00100000:0.0:1789762872.091867:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345334053945 time 345334153179 state 0x2 ref 0 00000100:00100000:0.0:1789762872.094798:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762872.094868:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067457664 00000100:00100000:3.0:1789762872.094882:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067457664 00000100:00100000:3.0:1789762872.094978:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345334053945 time 345334153179 state 0x0 ref 1 00000100:00100000:3.0:1789762872.094993:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345384153179 time 345334153179 state 0x1 ref 1 00000100:00100000:1.0:1789762872.137278:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345384153179 time 345384304563 state 0x0 ref 1 00000100:00100000:1.0:1789762872.137363:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067457664:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762872.137418:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176813 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.137439:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762872.137444:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762872.137452:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176813 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.137467:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176813 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.141947:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176813 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.141966:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176813 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.141979:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176813 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.142000:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762872.142006:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176813 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.142024:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176813 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.142046:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176813 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.142054:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176813 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762872.142128:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067457664:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4821us (47330us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762872.142140:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 690 00000100:00100000:1.0:1789762872.142153:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345384153179 time 345384304563 state 0x2 ref 0 00000100:00100000:3.0:1789762872.144613:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762872.144683:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067457792 00000100:00100000:1.0:1789762872.144697:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067457792 00000100:00100000:1.0:1789762872.144749:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345384153179 time 345384304563 state 0x0 ref 1 00000100:00100000:1.0:1789762872.144765:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345434304563 time 345384304563 state 0x1 ref 1 00000100:00100000:1.0:1789762872.187371:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345434304563 time 345434424235 state 0x0 ref 1 00000100:00100000:1.0:1789762872.187404:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067457792:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762872.187459:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17681a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.187480:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762872.187486:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762872.187494:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17681a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.187510:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17681a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.191605:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17681a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.191625:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17681a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.191641:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17681a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.191654:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762872.191663:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17681a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.191680:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17681a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.191723:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17681a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.191734:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f17681a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762872.191816:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067457792:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4420us (47202us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762872.191832:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 691 00000100:00100000:1.0:1789762872.191849:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345434304563 time 345434424235 state 0x2 ref 0 00000100:00100000:3.0:1789762872.193970:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762872.194088:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067457920 00000100:00100000:0.0:1789762872.194102:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067457920 00000100:00100000:0.0:1789762872.194178:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345434304563 time 345434424235 state 0x0 ref 1 00000100:00100000:0.0:1789762872.194198:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345484424235 time 345434424235 state 0x1 ref 1 00000100:00100000:0.0:1789762872.237454:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345484424235 time 345484518881 state 0x0 ref 1 00000100:00100000:0.0:1789762872.237481:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067457920:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762872.237546:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176821 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.237571:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762872.237579:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762872.237589:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176821 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.237624:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176821 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.241681:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176821 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.241700:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176821 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.241716:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176821 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.241729:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762872.241737:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176821 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.241753:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176821 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.241808:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176821 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.241818:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176821 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.241889:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067457920:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4415us (47923us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762872.241901:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 692 00000100:00100000:3.0:1789762872.241912:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345484424235 time 345484518881 state 0x2 ref 0 00000100:00100000:3.0:1789762872.267052:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762872.267193:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067458048 00000100:00100000:2.0:1789762872.267206:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067458048 00000100:00100000:2.0:1789762872.267300:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 55 00000100:00100000:2.0:1789762872.267341:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:6357:x1876702067458048:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762872.267382:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762872.267438:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176557 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8f6 expref: 6 pid: 8054 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.267480:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762872.267556:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:6357:x1876702067458048:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 200us (508us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762872.267566:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 55 00000100:00100000:2.0:1789762872.276738:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762872.276833:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067458176 00000100:00100000:0.0:1789762872.276844:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067458176 00000100:00100000:0.0:1789762872.276975:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 287 00000100:00080000:3.0:1789762872.276985:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762872.277014:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:13087:x1876702067458176:12345-192.168.203.37@tcp:101:bash.0 00010000:00010000:0.0:1789762872.277037:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762872.277059:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176828 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c8fd expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.277164:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f17682f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.277190:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f17682f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.277211:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f17682f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762872.277293:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 16384 00000004:00200000:0.0:1789762872.277359:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762872.277407:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f17682f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.277452:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176828 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8fd expref: 11 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.277475:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f17682f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8fd expref: 10 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.277486:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:0.0:1789762872.277491:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762872.277495:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca315800, rc 0) 00000100:00100000:0.0:1789762872.277563:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:13087:x1876702067458176:12345-192.168.203.37@tcp:101:bash.0 Request processed in 576us (828us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762872.277571:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 287 00000100:00100000:0.0:1789762872.281662:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762872.281738:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067458432 00000100:00100000:3.0:1789762872.281749:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067458432 00000100:00100000:3.0:1789762872.281806:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 288 00000100:00100000:3.0:1789762872.281818:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:14626:x1876702067458432:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:3.0:1789762872.281833:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762872.281849:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176836 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c904 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762872.281994:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 16384 00010000:00010000:3.0:1789762872.282136:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176836 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c904 expref: 12 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.282188:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762872.282192:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762872.282198:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176836 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c904 expref: 12 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.282215:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5933e00, rc 0) 00000100:00100000:3.0:1789762872.282283:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:14626:x1876702067458432:12345-192.168.203.37@tcp:101:rm.0 Request processed in 466us (624us total) trans 4294967418 rc 301/301 00000100:00100000:3.0:1789762872.282291:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 288 00000100:00100000:3.0:1789762872.284286:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762872.284406:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067458560 00000100:00100000:0.0:1789762872.284416:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067458560 00000100:00100000:0.0:1789762872.284500:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 80 00000100:00100000:0.0:1789762872.284520:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:14626:x1876702067458560:12345-192.168.203.37@tcp:37:rm.0 00000100:00100000:0.0:1789762872.288403:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:14626:x1876702067458560:12345-192.168.203.37@tcp:37:rm.0 Request processed in 3883us (4113us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762872.288422:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 80 00000100:00100000:2.0:1789762872.290219:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762872.290340:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067458816 00000100:00100000:0.0:1789762872.290350:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067458816 00000100:00100000:0.0:1789762872.290398:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 81 00000100:00100000:0.0:1789762872.290429:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:14626:x1876702067458816:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:0.0:1789762872.290653:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:14626:x1876702067458816:12345-192.168.203.37@tcp:35:rm.0 Request processed in 241us (437us total) trans 4294967419 rc 0/0 00000100:00100000:0.0:1789762872.290668:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 81 00000100:00100000:3.0:1789762872.291825:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762872.291894:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067459072 00000100:00100000:2.0:1789762872.291902:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067459072 00000100:00100000:2.0:1789762872.291984:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 289 00000100:00100000:2.0:1789762872.291997:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:14626:x1876702067459072:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:2.0:1789762872.292013:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762872.292028:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176844 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c90b expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.292104:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17684b lrc: 3/1,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.292118:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17684b lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.292140:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17684b lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762872.292170:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 16384 00010000:00010000:2.0:1789762872.292279:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f176844 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c90b expref: 15 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.292300:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17684b lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c90b expref: 14 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.292332:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762872.292336:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1789762872.292339:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768600, rc 0) 00000100:00100000:2.0:1789762872.292421:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14626:x1876702067459072:12345-192.168.203.37@tcp:101:rm.0 Request processed in 422us (595us total) trans 4294967420 rc 0/0 00000100:00100000:2.0:1789762872.292440:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 289 00000100:00100000:2.0:1789762872.293672:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762872.293768:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067459200 00000100:00100000:1.0:1789762872.293777:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067459200 00000100:00100000:1.0:1789762872.293822:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 290 00000100:00100000:1.0:1789762872.293833:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14626:x1876702067459200:12345-192.168.203.37@tcp:41:rm.0 00000004:00000024:1.0:1789762872.293891:0:6420:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7597981696 avail 7586447360 00000100:00100000:1.0:1789762872.294015:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14626:x1876702067459200:12345-192.168.203.37@tcp:41:rm.0 Request processed in 181us (342us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762872.294025:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 290 00000100:00100000:0.0:1789762872.295258:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762872.295368:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067459456 00000100:00100000:2.0:1789762872.295379:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067459456 00000100:00100000:2.0:1789762872.295434:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 82 00000100:00100000:2.0:1789762872.295454:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbf050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14626:x1876702067459456:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:2.0:1789762872.295753:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbf050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:14626:x1876702067459456:12345-192.168.203.37@tcp:35:rm.0 Request processed in 300us (498us total) trans 4294967421 rc 0/0 00000100:00100000:2.0:1789762872.295763:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 82 00000100:00100000:3.0:1789762872.296765:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762872.296821:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067459712 00000100:00100000:2.0:1789762872.296828:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067459712 00000100:00100000:2.0:1789762872.296878:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 291 00000100:00100000:2.0:1789762872.296887:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:14626:x1876702067459712:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:2.0:1789762872.296981:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789762872.297011:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273d200/0xdf5a7ac59f176549 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8ef expref: 15 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.297037:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f17626a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8e8 expref: 14 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.297052:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff9c37e125a200) 00010000:00010000:2.0:1789762872.297067:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f17682f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8fd expref: 13 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.297078:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762872.297082:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762872.297118:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176859 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.297134:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176859 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.297148:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176859 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.297172:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176860 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.297187:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176860 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.297214:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176860 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.297713:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176867 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.297727:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176867 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.297737:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176867 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789762872.299504:0:6419:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].7, 64 off8640 00000004:00080000:2.0:1789762872.299526:0:6419:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 345546606037ns, next at 346546601987ns, rc = 0 00010000:00010000:2.0:1789762872.299602:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f17686e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299614:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f17686e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299624:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f17686e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299632:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f17686e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299639:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f17686e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299646:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f17686e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299688:0:6419:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762872.299705:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176867 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299713:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176867 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299721:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176867 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299728:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762872.299733:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176867 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299741:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176867 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.299753:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762872.299754:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff9c37d3beea00) 00010000:00010000:2.0:1789762872.299765:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762872.299771:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f17686e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299771:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176867 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299806:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769200/0xdf5a7ac59f176867 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.299807:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762872.299818:0:6408:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f17686e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.299833:0:6408:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f17686e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299834:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176859 lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299856:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176859 lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.299863:0:6408:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff9c37d3beea00) 00010000:00010000:2.0:1789762872.299865:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176859 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.299869:0:6408:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789762872.299875:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f17686e lrc: 1/0,0 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299901:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176859 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.299902:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769800/0xdf5a7ac59f17686e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299949:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762872.299956:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176859 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.299985:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176859 lrc: 5/0,0 mode: CW/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.300001:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762872.300005:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762872.300011:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176859 lrc: 3/0,0 mode: --/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.300015:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762872.300023:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762872.300026:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f176859 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.300029:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00010000:2.0:1789762872.300050:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d9d537f9 reply state 0000000073d23158 transno 4294967422 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176860 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.300066:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176860 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762872.300074:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x4009f790 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762872.300077:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762872.300081:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762872.300168:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14626:x1876702067459712:12345-192.168.203.37@tcp:36:rm.0 Request processed in 3276us (3402us total) trans 4294967422 rc 0/0 00000100:00100000:2.0:1789762872.300184:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 291 00000100:00100000:0.0:1789762872.301675:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762872.301747:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067459968 00000100:00100000:2.0:1789762872.301756:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067459968 00000100:00100000:2.0:1789762872.301800:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 83 00000100:00100000:2.0:1789762872.301811:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14626:x1876702067459968:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:2.0:1789762872.301981:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:14626:x1876702067459968:12345-192.168.203.37@tcp:35:rm.0 Request processed in 168us (306us total) trans 4294967423 rc 0/0 00000100:00100000:2.0:1789762872.301991:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 83 00000100:00100000:0.0:1789762872.303022:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762872.303091:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067460224 00000100:00100000:3.0:1789762872.303102:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067460224 00000100:00100000:3.0:1789762872.303167:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 292 00000100:00100000:3.0:1789762872.303186:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:14626:x1876702067460224:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:3.0:1789762872.303225:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1789762872.303244:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17684b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c90b expref: 15 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.303267:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d273da00/0xdf5a7ac59f17624e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8e1 expref: 14 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.303281:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:3.0:1789762872.303295:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f1761ec lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c8cc expref: 13 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.303348:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762872.303354:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762872.303375:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176875 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.303393:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176875 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.303408:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176875 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.303429:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f17687c lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.303443:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f17687c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.303455:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f17687c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.303865:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f176883 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.303878:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f176883 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.303890:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f176883 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762872.304795:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 16384 00010000:00010000:3.0:1789762872.304810:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f176883 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.304828:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f176883 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.304835:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f176883 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.304841:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762872.304846:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f176883 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.304854:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f176883 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.304865:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff9c37c72ba200) 00010000:00010000:3.0:1789762872.304868:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789762872.304872:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f176883 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.304878:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f176883 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.304901:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176875 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.304908:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176875 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.304943:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176875 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.304972:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176875 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.304996:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762872.305003:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176875 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.305033:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176875 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.305051:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762872.305056:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762872.305063:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176875 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.305075:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176875 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762872.305112:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000062151dd6 reply state 000000001519f4a6 transno 4294967424 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f17687c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.305128:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f17687c lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.305136:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72bad00) 00010000:00010000:3.0:1789762872.305139:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1789762872.305142:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762872.305203:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:14626:x1876702067460224:12345-192.168.203.37@tcp:36:rm.0 Request processed in 2023us (2182us total) trans 4294967424 rc 0/0 00000100:00100000:3.0:1789762872.305212:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 292 00000001:00080000:2.0:1789762872.333904:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967911 is committed 00000100:00100000:3.0:1789762872.832071:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074703488:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762872.832136:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762872.832177:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074703488 00000100:00100000:1.0:1789762872.832185:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074703488 00000100:00100000:1.0:1789762872.832231:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:1.0:1789762872.832242:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3425:x1876702074703488:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789762872.832275:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:1.0:1789762872.832347:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3425:x1876702074703488:12345-0@lo:41:osp-pre-0-1.0 Request processed in 106us (209us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762872.832357:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00100000:3.0:1789762872.832381:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074703488:0@lo:41:osp-pre-0-1.0 00000001:00080000:2.0:1789762872.842506:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967419 is committed 00000001:00080000:2.0:1789762872.842521:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967417 is committed 00000001:00080000:2.0:1789762872.842536:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967423 is committed 00000004:00080000:2.0:1789762872.842543:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 11 committed 00010000:00010000:0.0:1789762872.842561:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176860 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.842574:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176860 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762872.842577:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967424 is committed 00010000:00010000:0.0:1789762872.842590:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176860 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.842599:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762872.842603:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f17687c lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.842606:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176860 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.842618:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f17687c lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.842624:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176860 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.842628:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f17687c lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.842642:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762872.842647:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f17687c lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.842656:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f17687c lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.842657:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x4009f790 (ffff9c37d3beed00) 00000100:00100000:2.0:1789762872.842661:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074703616:0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1789762872.842668:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762872.842677:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176860 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.842685:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176860 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x16:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.842690:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72bad00) 00010000:00010000:1.0:1789762872.842695:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762872.842703:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f17687c lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.842714:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933c00/0xdf5a7ac59f17687c lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762872.842736:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762872.842770:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074703616 00000100:00100000:0.0:1789762872.842776:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074703616 00000100:00100000:0.0:1789762872.842817:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:0.0:1789762872.842824:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074703616:12345-0@lo:6:osp-syn-0-0.0 00000100:00080000:3.0:1789762872.842826:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00080000:0.0:1789762872.842844:0:8054:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:7 count 1 00002000:00010000:0.0:1789762872.842866:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x7:0x0] initial LVB size: 104857600, mtime: 0x6aad9d33, atime: 0x0, ctime: 0x6aad9d33, blocks: 0x3200b 00010000:00010000:0.0:1789762872.842872:0:8054:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17688a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.842883:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=346s v=(0 0) 00010000:00010000:0.0:1789762872.842887:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762872.842892:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17688a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.842902:0:8054:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17688a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.842908:0:8054:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17688a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.842913:0:8054:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17688a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.842950:0:8054:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762872.842963:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762872.842971:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17688a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.842978:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762872.842982:0:7217:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17688a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.842988:0:7217:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17688a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.843000:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17688a lrc: 1/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.843009:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17688a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.843024:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762872.843026:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762872.843028:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762872.843382:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074703616:12345-0@lo:6:osp-syn-0-0.0 Request processed in 558us (647us total) trans 4294967912 rc 0/0 00000100:00100000:0.0:1789762872.843395:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000004:00080000:2.0:1789762872.843434:0:3426:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37d1ae0000/2, rc 0, transno 4294967912 00000100:00100000:2.0:1789762872.843446:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074703616:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1789762872.960064:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1789762873.019558:0:16645:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 346266636957ns, next at 346546601987ns, rc = 0 00000004:00080000:2.0:1789762873.019675:0:16645:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 346266757079ns, next at 338887174414ns, rc = 0 00000004:00080000:2.0:1789762873.019819:0:16645:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 346266900197ns, next at 0ns, rc = 0 00000004:00080000:2.0:1789762873.020084:0:16645:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 346267164707ns, next at 338887170361ns, rc = 0 00000004:00080000:0.0:1789762873.022285:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 12 committed 00000004:00080000:0.0:1789762873.023589:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 6 committed 00000004:00080000:1.0:1789762873.031816:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 17 committed 00000004:00080000:3.0:1789762873.032872:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 17 committed 00000100:00100000:3.0:1789762873.800012:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762873.800049:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762873.800074:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067460480 00000100:00100000:3.0:1789762873.800080:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762873.800085:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067460480 00000100:00100000:1.0:1789762873.800109:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067460352 00000100:00100000:1.0:1789762873.800118:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067460352 00000100:00100000:3.0:1789762873.800120:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762873.800156:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067460608 00000100:00100000:0.0:1789762873.800161:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067460608 00000100:00100000:2.0:1789762873.800180:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 498 00000100:00100000:3.0:1789762873.800180:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762873.800192:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 361 00000100:00100000:1.0:1789762873.800213:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1553050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067460352:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762873.800226:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 499 00000100:00100000:2.0:1789762873.800228:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067460480:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762873.800234:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762873.800250:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067460608:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762873.800304:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762873.800312:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1553050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067460352:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 91us (241us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762873.800318:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067460608:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 75us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762873.800327:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 361 00000100:00100000:0.0:1789762873.800332:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 499 00000100:00100000:2.0:1789762873.800347:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067460480:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 142us (338us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762873.800359:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067460992 00000100:00100000:0.0:1789762873.800365:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067460992 00000100:00100000:2.0:1789762873.800380:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 498 00000100:00100000:0.0:1789762873.800406:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067461120 00000100:00100000:0.0:1789762873.800410:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067461120 00000100:00100000:2.0:1789762873.800428:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 500 00000100:00100000:1.0:1789762873.800429:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067460736 00000100:00100000:1.0:1789762873.800436:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067460736 00000100:00100000:2.0:1789762873.800445:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067460992:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762873.800455:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 501 00000100:00100000:0.0:1789762873.800467:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067461120:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762873.800484:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067460864 00000100:00100000:1.0:1789762873.800488:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067460864 00000100:00100000:2.0:1789762873.800519:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067460992:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 70us (288us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762873.800520:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067461120:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 45us (218us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762873.800535:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 200 00000100:00100000:1.0:1789762873.800558:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067460736:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762873.800564:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 500 00000100:00100000:0.0:1789762873.800577:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 501 00000100:00100000:3.0:1789762873.800588:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 201 00000100:00100000:1.0:1789762873.800602:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067460736:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 48us (487us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762873.800605:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067460864:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762873.800613:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 200 00000100:00080000:2.0:1789762873.800618:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762873.800628:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762873.800635:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762873.800650:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762873.800665:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067460864:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 66us (497us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762873.800677:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 201 00010000:00010000:1.0:1789762873.984151:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762874.880047:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762874.880070:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff615f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074704128:0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.880077:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8a9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074704000:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762874.880156:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762874.880157:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762874.880200:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074703872:0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.880221:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074704000 00000100:00100000:1.0:1789762874.880230:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074704000 00000100:00100000:2.0:1789762874.880234:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074703744:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762874.880238:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff617800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074704640:0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.880311:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:0.0:1789762874.880320:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762874.880321:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762874.880325:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762874.880335:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7ada450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074704000:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789762874.880372:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762874.880403:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff614e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074704512:0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.880404:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff614700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074705280:0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.880408:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074704768:0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.880427:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7ada450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074704000:12345-0@lo:13:osp-pre-1-0.0 Request processed in 96us (280us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762874.880435:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00000100:00100000:0.0:1789762874.880478:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762874.880486:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074703744 00000100:00100000:3.0:1789762874.880491:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074703744 00000100:00100000:2.0:1789762874.880502:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074704256:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762874.880534:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff615880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074705024:0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.880545:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:2.0:1789762874.880548:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762874.880564:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca558a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074703744:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762874.880580:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff614000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074704384:0@lo:400:kworker.0 00002000:00000024:3.0:1789762874.880596:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762874.880638:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762874.880639:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762874.880651:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca558a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074703744:12345-0@lo:13:osp-pre-0-1.0 Request processed in 95us (341us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762874.880662:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:3.0:1789762874.880686:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074704256 00000100:00100000:3.0:1789762874.880690:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074704256 00000100:00100000:0.0:1789762874.880696:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff615180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074705536:0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.880697:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff617100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074704896:0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.880717:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:3.0:1789762874.880730:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074704256:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762874.880745:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3754950656 avail 3752853504 00000100:00100000:1.0:1789762874.880774:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074704128 00000100:00100000:2.0:1789762874.880781:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762874.880783:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762874.880785:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074704128 00000100:00100000:3.0:1789762874.880796:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074704256:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (250us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762874.880808:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:2.0:1789762874.880828:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074705408:0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.880833:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074704512 00000100:00100000:1.0:1789762874.880837:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074704512 00000100:00100000:3.0:1789762874.880854:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074705024 00000100:00100000:3.0:1789762874.880860:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074705024 00000100:00100000:2.0:1789762874.880877:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762874.880882:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074703872 00000100:00100000:1.0:1789762874.880882:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074704896 00000100:00100000:3.0:1789762874.880892:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 502 00000100:00100000:1.0:1789762874.880894:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074704896 00000100:00100000:0.0:1789762874.880896:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074703872 00000100:00100000:3.0:1789762874.880904:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074704128:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.880990:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 503 00000100:00100000:3.0:1789762874.881003:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074704128:12345-0@lo:400:kworker.0 Request processed in 99us (856us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762874.881004:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074704512:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.881004:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074704256:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762874.881029:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 502 00000100:00100000:0.0:1789762874.881030:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 362 00000100:00100000:2.0:1789762874.881055:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074704640 00000100:00100000:0.0:1789762874.881056:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1550450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074703872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.881061:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074704640 00000100:00100000:3.0:1789762874.881064:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 504 00000100:00100000:1.0:1789762874.881070:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074704512:12345-0@lo:400:kworker.0 Request processed in 67us (593us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762874.881081:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074705024:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.881086:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 503 00000100:00100000:2.0:1789762874.881108:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074704384 00000100:00100000:1.0:1789762874.881112:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 505 00000100:00100000:2.0:1789762874.881113:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074704384 00000100:00100000:1.0:1789762874.881126:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074704896:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.881145:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074705024:12345-0@lo:400:kworker.0 Request processed in 64us (521us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762874.881154:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 504 00000100:00100000:0.0:1789762874.881162:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1550450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074703872:12345-0@lo:400:kworker.0 Request processed in 113us (855us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762874.881164:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 337 00000100:00100000:1.0:1789762874.881180:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074704896:12345-0@lo:400:kworker.0 Request processed in 56us (408us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762874.881189:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 362 00000100:00100000:2.0:1789762874.881196:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfedc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074704640:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.881199:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762874.881207:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 505 00000100:00100000:1.0:1789762874.881229:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074705280 00000100:00100000:1.0:1789762874.881232:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074705280 00000100:00100000:3.0:1789762874.881234:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff615500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074705664:0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.881263:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfedc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074704640:12345-0@lo:400:kworker.0 Request processed in 81us (951us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762874.881278:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 337 00000100:00100000:3.0:1789762874.881297:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762874.881298:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 506 00000100:00100000:1.0:1789762874.881315:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074705280:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.881321:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 338 00000100:00100000:2.0:1789762874.881337:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074704384:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762874.881338:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074705536 00000100:00100000:0.0:1789762874.881345:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074705536 00000100:00100000:1.0:1789762874.881371:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074705280:12345-0@lo:400:kworker.0 Request processed in 61us (184us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762874.881371:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8a9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074704000:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762874.881389:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 506 00000100:00100000:2.0:1789762874.881400:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074704384:12345-0@lo:400:kworker.0 Request processed in 67us (774us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762874.881412:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 338 00000100:00100000:0.0:1789762874.881416:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074705408 00000100:00080000:1.0:1789762874.881426:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762874.881426:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff617800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074704640:0@lo:400:kworker.0 00000100:00100000:0.0:1789762874.881430:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074705408 00000100:00080000:1.0:1789762874.881434:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762874.881442:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789762874.881447:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762874.881447:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074705664 00000100:00100000:2.0:1789762874.881454:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074705664 00000100:00100000:3.0:1789762874.881454:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff614700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074705280:0@lo:400:kworker.0 00000100:00100000:0.0:1789762874.881491:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:2.0:1789762874.881499:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:0.0:1789762874.881504:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074705536:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.881512:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074705408:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.881517:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:3.0:1789762874.881534:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074705664:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.881553:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762874.881590:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074705536:12345-0@lo:400:kworker.0 Request processed in 90us (819us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762874.881599:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074705408:12345-0@lo:400:kworker.0 Request processed in 88us (723us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762874.881607:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:1.0:1789762874.881615:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074705152:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762874.881624:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:3.0:1789762874.881632:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074705664:12345-0@lo:400:kworker.0 Request processed in 98us (348us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762874.881646:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074704768 00000100:00100000:3.0:1789762874.881650:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:0.0:1789762874.881653:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074704768 00000100:00100000:1.0:1789762874.881660:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762874.881699:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967417 gen 2 00000100:00100000:1.0:1789762874.881701:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff614380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074705792:0@lo:400:kworker.0 00000004:00080000:2.0:1789762874.881705:0:3427:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967416 committed transno 4294967417 00000100:00100000:3.0:1789762874.881722:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff615500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074705664:0@lo:400:kworker.0 00000100:00100000:0.0:1789762874.881728:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 507 00000004:00080000:2.0:1789762874.881737:0:3427:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967417 committed transno 4294967417 00000100:00100000:1.0:1789762874.881746:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762874.881747:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074704768:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.881756:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff614000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074704384:0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.881785:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff617100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074704896:0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.881802:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff615f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074704128:0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.881808:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074703872:0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.881823:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294967911 gen 2 00000100:00100000:0.0:1789762874.881826:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074704768:12345-0@lo:400:kworker.0 Request processed in 84us (272us total) trans 0 rc 0/0 00000004:00080000:2.0:1789762874.881833:0:3427:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c36ca55b800, transno 4294967911 00000100:00100000:3.0:1789762874.881841:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff614e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074704512:0@lo:400:kworker.0 00000100:00100000:0.0:1789762874.881843:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 507 00000100:00100000:2.0:1789762874.881856:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074705408:0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.881862:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074703744:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762874.881884:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff615880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074705024:0@lo:400:kworker.0 00000100:00080000:0.0:1789762874.881901:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762874.881906:0:3424:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294967399 gen 2 00000004:00080000:3.0:1789762874.881914:0:3424:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c36c9ffc380, transno 4294967399 00000100:00100000:3.0:1789762874.881972:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff615180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074705536:0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.881975:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074704768:0@lo:400:kworker.0 00000100:00100000:0.0:1789762874.881994:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074705152 00000100:00100000:0.0:1789762874.882003:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074705152 00000100:00100000:1.0:1789762874.882016:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074705792 00000100:00100000:1.0:1789762874.882021:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074705792 00000100:00100000:0.0:1789762874.882057:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:1.0:1789762874.882063:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:1.0:1789762874.882079:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff617800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074705792:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762874.882082:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074705152:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789762874.882105:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762874.882132:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff617800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074705792:12345-0@lo:400:kworker.0 Request processed in 60us (394us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762874.882141:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:3.0:1789762874.882172:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff614380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074705792:0@lo:400:kworker.0 00000100:00100000:0.0:1789762874.882174:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074705152:12345-0@lo:13:osp-pre-1-1.0 Request processed in 106us (516us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762874.882190:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:3.0:1789762874.882222:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074705152:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1789762875.008059:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762876.032030:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762876.928137:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074705920:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762876.928208:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762876.928295:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074705920 00000100:00100000:2.0:1789762876.928306:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074705920 00000100:00100000:2.0:1789762876.928378:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 339 00000100:00100000:2.0:1789762876.928396:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfeea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074705920:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762876.928436:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762876.928502:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfeea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074705920:12345-0@lo:41:osp-pre-1-0.0 Request processed in 113us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762876.928513:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 339 00000100:00100000:3.0:1789762876.928541:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074705920:0@lo:41:osp-pre-1-0.0 00010000:00010000:3.0:1789762877.056054:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762877.461305:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967912 is committed 00000100:00100000:3.0:1789762877.952061:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789762877.952082:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074706048:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762877.952137:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762877.952187:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074706048 00000100:00100000:2.0:1789762877.952197:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074706048 00000100:00100000:2.0:1789762877.952244:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 340 00000100:00100000:2.0:1789762877.952254:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074706048:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762877.952303:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762877.952355:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfed500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074706048:12345-0@lo:41:osp-pre-0-1.0 Request processed in 101us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762877.952363:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 340 00000100:00100000:3.0:1789762877.952395:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074706048:0@lo:41:osp-pre-0-1.0 00010000:00010000:1.0:1789762878.080048:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762878.919885:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762878.919955:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762878.919981:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762878.919985:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067461248 00000100:00100000:0.0:1789762878.919996:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067461248 00000100:00100000:1.0:1789762878.920014:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762878.920027:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067461376 00000100:00100000:3.0:1789762878.920035:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067461632 00000100:00100000:2.0:1789762878.920036:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067461376 00000100:00100000:3.0:1789762878.920044:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067461632 00000100:00100000:0.0:1789762878.920050:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 363 00000100:00100000:1.0:1789762878.920051:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762878.920065:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1550c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067461248:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762878.920086:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762878.920094:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067461504 00000100:00100000:2.0:1789762878.920098:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067461504 00000100:00100000:3.0:1789762878.920101:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067461760 00000100:00100000:3.0:1789762878.920106:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067461760 00000100:00100000:2.0:1789762878.920128:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067461888 00000100:00100000:0.0:1789762878.920133:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1550c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067461248:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 71us (256us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762878.920134:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067461888 00000100:00100000:0.0:1789762878.920145:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 363 00000100:00100000:3.0:1789762878.920149:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 206 00000100:00100000:3.0:1789762878.920163:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067461632:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762878.920172:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 508 00000100:00100000:1.0:1789762878.920177:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00080000:0.0:1789762878.920185:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762878.920188:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067461376:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:0.0:1789762878.920191:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762878.920194:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762878.920196:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762878.920206:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067461632:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 45us (222us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762878.920215:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067462016 00000100:00100000:3.0:1789762878.920216:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 206 00000100:00100000:0.0:1789762878.920219:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067462016 00000100:00100000:2.0:1789762878.920238:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067461376:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 54us (280us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762878.920246:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 508 00000100:00100000:0.0:1789762878.920268:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 509 00000100:00100000:0.0:1789762878.920281:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067461504:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762878.920290:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 510 00000100:00100000:3.0:1789762878.920300:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 207 00000100:00100000:2.0:1789762878.920306:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067461888:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762878.920313:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067461504:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 34us (298us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762878.920316:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067461760:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762878.920324:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 509 00000100:00100000:1.0:1789762878.920345:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 511 00000100:00100000:2.0:1789762878.920352:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067461888:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 52us (304us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762878.920359:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762878.920360:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067462016:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762878.920361:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 510 00000100:00100000:1.0:1789762878.920403:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067462016:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 49us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762878.920412:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 511 00000100:00100000:3.0:1789762878.920426:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067461760:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 115us (339us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762878.920473:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 207 00010000:00010000:1.0:1789762879.104018:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762880.000060:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fcfed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074706176:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762880.000061:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1abad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074706432:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762880.000132:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762880.000143:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074706304:0@lo:400:kworker.0 00000100:00100000:2.0:1789762880.000167:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762880.000193:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762880.000197:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1aba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074706816:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762880.000200:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff614000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074706944:0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.000223:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff617100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074706688:0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.000255:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762880.000269:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762880.000269:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762880.000287:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074707328:0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.000301:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ae3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074707200:0@lo:400:kworker.0 00000100:00100000:2.0:1789762880.000303:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074707456:0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.000327:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762880.000345:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762880.000354:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074707840:0@lo:400:kworker.0 00000100:00100000:2.0:1789762880.000358:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074706944 00000100:00100000:2.0:1789762880.000366:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074706944 00000100:00100000:1.0:1789762880.000384:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074707712:0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.000392:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762880.000420:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074707328 00000100:00100000:1.0:1789762880.000424:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762880.000425:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074707328 00000100:00100000:3.0:1789762880.000438:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074706176 00000100:00100000:3.0:1789762880.000444:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074706176 00000100:00100000:1.0:1789762880.000456:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f60a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074708224:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762880.000469:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074707200 00000100:00100000:2.0:1789762880.000473:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074707200 00000100:00100000:1.0:1789762880.000488:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762880.000490:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074706432 00000100:00100000:3.0:1789762880.000496:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074706432 00000100:00100000:2.0:1789762880.000513:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 512 00000100:00100000:1.0:1789762880.000522:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:3.0:1789762880.000524:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074706816 00000100:00100000:2.0:1789762880.000528:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074706944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.000529:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074706816 00000100:00100000:1.0:1789762880.000530:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074706560:0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.000554:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074708224 00000100:00100000:3.0:1789762880.000558:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074708224 00000100:00100000:1.0:1789762880.000568:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762880.000585:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:1.0:1789762880.000590:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff615500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074707072:0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.000597:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074706176:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762880.000599:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074706944:12345-0@lo:400:kworker.0 Request processed in 75us (331us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762880.000612:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 512 00000100:00100000:1.0:1789762880.000624:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:3.0:1789762880.000628:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762880.000635:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074706560 00000100:00100000:2.0:1789762880.000647:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074706560 00000100:00100000:1.0:1789762880.000650:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074707584:0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.000675:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074706176:12345-0@lo:13:osp-pre-1-1.0 Request processed in 82us (543us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762880.000679:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 513 00000100:00100000:3.0:1789762880.000688:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00100000:1.0:1789762880.000694:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762880.000694:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074707328:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.000712:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:1.0:1789762880.000716:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a08a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074708096:0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.000723:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074706432:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762880.000736:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074707328:12345-0@lo:400:kworker.0 Request processed in 46us (408us total) trans 0 rc 0/0 00002000:00000024:3.0:1789762880.000743:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762880.000745:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762880.000754:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 513 00000100:00100000:2.0:1789762880.000778:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074707584 00000100:00100000:2.0:1789762880.000782:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074707584 00000100:00100000:3.0:1789762880.000789:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074706432:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (622us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762880.000795:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00100000:1.0:1789762880.000808:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:2.0:1789762880.000813:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 514 00000100:00100000:3.0:1789762880.000817:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789762880.000823:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074706816:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762880.000823:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074707200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.000830:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074708224:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789762880.000837:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3859808256 avail 3857711104 00002000:00000024:3.0:1789762880.000842:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762880.000881:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074706816:12345-0@lo:13:osp-pre-0-1.0 Request processed in 61us (634us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762880.000881:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074707200:12345-0@lo:400:kworker.0 Request processed in 57us (534us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762880.000888:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074708224:12345-0@lo:13:osp-pre-1-0.0 Request processed in 60us (398us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762880.000894:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:2.0:1789762880.000894:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 514 00000100:00100000:3.0:1789762880.000907:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:2.0:1789762880.000947:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 515 00000100:00100000:1.0:1789762880.000949:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074706304 00000100:00100000:1.0:1789762880.000953:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074706304 00000100:00100000:2.0:1789762880.000961:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074706560:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.001001:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:2.0:1789762880.001007:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074706560:12345-0@lo:400:kworker.0 Request processed in 51us (441us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762880.001012:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1552050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074706304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762880.001014:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 515 00000100:00100000:2.0:1789762880.001040:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 516 00000100:00100000:1.0:1789762880.001048:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1552050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074706304:12345-0@lo:400:kworker.0 Request processed in 40us (855us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762880.001048:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074707584:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.001057:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:2.0:1789762880.001083:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074707584:12345-0@lo:400:kworker.0 Request processed in 35us (395us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762880.001091:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 516 00000100:00100000:1.0:1789762880.001100:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074706688 00000100:00100000:1.0:1789762880.001104:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074706688 00000100:00080000:2.0:1789762880.001119:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762880.001126:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762880.001129:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762880.001132:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762880.001136:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074707072 00000100:00100000:1.0:1789762880.001139:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074707072 00000100:00100000:3.0:1789762880.001162:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074707328:0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.001168:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 341 00000100:00100000:2.0:1789762880.001177:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 342 00000100:00100000:1.0:1789762880.001178:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074706688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.001184:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fcfed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074706176:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762880.001188:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074707072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.001213:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1aba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074706816:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762880.001223:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074706688:12345-0@lo:400:kworker.0 Request processed in 45us (955us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762880.001224:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074707072:12345-0@lo:400:kworker.0 Request processed in 40us (601us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762880.001233:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 342 00000100:00100000:1.0:1789762880.001238:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 341 00000100:00100000:1.0:1789762880.001292:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074707840 00000100:00100000:1.0:1789762880.001296:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074707840 00000100:00100000:3.0:1789762880.001313:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074707712 00000100:00100000:3.0:1789762880.001319:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074707712 00000100:00100000:2.0:1789762880.001333:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074708096 00000100:00100000:1.0:1789762880.001336:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:2.0:1789762880.001339:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074708096 00000100:00100000:1.0:1789762880.001347:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074707840:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.001361:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:2.0:1789762880.001369:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:3.0:1789762880.001374:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074707712:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762880.001383:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074708096:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.001397:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074707840:12345-0@lo:400:kworker.0 Request processed in 54us (1008us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762880.001405:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:3.0:1789762880.001424:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074707712:12345-0@lo:400:kworker.0 Request processed in 53us (999us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762880.001432:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:2.0:1789762880.001440:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074708096:12345-0@lo:400:kworker.0 Request processed in 60us (694us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762880.001449:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074707840:0@lo:400:kworker.0 00000100:00100000:2.0:1789762880.001449:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00100000:3.0:1789762880.001467:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762880.001487:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074706560:0@lo:400:kworker.0 00000100:00100000:2.0:1789762880.001487:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074707456 00000100:00100000:2.0:1789762880.001491:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074707456 00000100:00100000:3.0:1789762880.001500:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7a0aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074707968:0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.001506:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff615500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074707072:0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.001526:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074707584:0@lo:400:kworker.0 00000100:00100000:2.0:1789762880.001528:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 517 00000100:00100000:3.0:1789762880.001535:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762880.001543:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a08a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074708096:0@lo:400:kworker.0 00000100:00100000:2.0:1789762880.001543:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074707456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.001563:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074706304:0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.001570:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967912 gen 2 00000004:00080000:3.0:1789762880.001576:0:3427:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37d1ae0000, transno 4294967912 00000100:00100000:1.0:1789762880.001583:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff617100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074706688:0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.001595:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1abad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074706432:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762880.001597:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074707456:12345-0@lo:400:kworker.0 Request processed in 57us (129us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762880.001604:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ae3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074707200:0@lo:400:kworker.0 00000100:00100000:2.0:1789762880.001607:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 517 00000100:00100000:3.0:1789762880.001618:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff614000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074706944:0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.001630:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074707712:0@lo:400:kworker.0 00000100:00080000:2.0:1789762880.001632:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762880.001660:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074707456:0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.001662:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f60a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074708224:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762880.001715:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074707968 00000100:00100000:1.0:1789762880.001719:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074707968 00000100:00100000:1.0:1789762880.001745:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:1.0:1789762880.001753:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074707968:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.001797:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074707968:12345-0@lo:400:kworker.0 Request processed in 44us (264us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762880.001803:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00100000:3.0:1789762880.001822:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7a0aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074707968:0@lo:400:kworker.0 00010000:00010000:3.0:1789762880.128015:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762881.152102:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762882.048122:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8aa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074708352:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762882.048208:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762882.048258:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074708352 00000100:00100000:1.0:1789762882.048268:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074708352 00000100:00100000:1.0:1789762882.048320:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 343 00000100:00100000:1.0:1789762882.048336:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074708352:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762882.048373:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762882.048436:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074708352:12345-0@lo:41:osp-pre-1-0.0 Request processed in 105us (227us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762882.048446:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 343 00000100:00100000:3.0:1789762882.048471:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8aa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074708352:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1789762882.176052:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762883.072087:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789762883.072107:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074708480:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789762883.072167:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762883.072210:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074708480 00000100:00100000:3.0:1789762883.072219:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074708480 00000100:00100000:3.0:1789762883.072285:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:3.0:1789762883.072299:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074708480:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789762883.072337:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789762883.072393:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074708480:12345-0@lo:41:osp-pre-0-1.0 Request processed in 96us (225us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762883.072400:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:0.0:1789762883.072432:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074708480:0@lo:41:osp-pre-0-1.0 00010000:00010000:1.0:1789762883.200086:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762884.040133:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762884.040170:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762884.040202:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762884.040208:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067462144 00000100:00100000:0.0:1789762884.040232:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067462144 00000100:00100000:3.0:1789762884.040236:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067462656 00000100:00100000:3.0:1789762884.040244:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067462656 00000100:00100000:1.0:1789762884.040246:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762884.040288:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762884.040307:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067462528 00000100:00100000:0.0:1789762884.040310:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 365 00000100:00100000:3.0:1789762884.040312:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067462528 00000100:00100000:1.0:1789762884.040321:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762884.040337:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1553c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067462144:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762884.040343:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067462272 00000100:00100000:3.0:1789762884.040355:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 212 00000100:00100000:2.0:1789762884.040357:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067462272 00000100:00100000:1.0:1789762884.040364:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762884.040368:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067462528:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762884.040412:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 175 00000100:00100000:1.0:1789762884.040427:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2394:x1876702067462656:12345-192.168.203.37@tcp:9:kworker.0 00000100:00100000:2.0:1789762884.040437:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067462400 00000100:00100000:0.0:1789762884.040440:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1553c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067462144:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 106us (311us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762884.040441:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067462528:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 68us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762884.040442:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067462400 00000020:00000080:1.0:1789762884.040458:0:9038:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xdf5a7ac59f17604f 00000100:00100000:0.0:1789762884.040468:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 365 00000100:00100000:3.0:1789762884.040469:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 212 00000100:00100000:2.0:1789762884.040472:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067462912 00000100:00080000:1.0:1789762884.040477:0:9038:0:(import.c:65:import_set_state_nolock()) ffff9c37d1023800 : changing import state from FULL to CLOSED 00000100:00100000:2.0:1789762884.040479:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067462912 00000100:00080000:3.0:1789762884.040512:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762884.040522:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762884.040531:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067462784 00000100:00080000:3.0:1789762884.040531:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762884.040537:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067462784 00000001:00000004:1.0:1789762884.040543:0:9038:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967399 00000100:00080000:3.0:1789762884.040548:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789762884.040556:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762884.040579:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 518 00000100:00100000:2.0:1789762884.040607:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067462272:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762884.040614:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 519 00000100:00100000:3.0:1789762884.040629:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f60a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067462400:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762884.040681:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067462272:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 79us (478us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762884.040690:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f60a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067462400:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 60us (396us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762884.040699:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 518 00000100:00100000:3.0:1789762884.040802:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 519 00000100:00100000:2.0:1789762884.040811:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 520 00000100:00100000:0.0:1789762884.040823:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 521 00000100:00100000:2.0:1789762884.040825:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067462912:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762884.040846:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7a08a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067462784:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762884.040868:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067462912:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 50us (550us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762884.040875:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 520 00000100:00100000:0.0:1789762884.040906:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7a08a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067462784:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 64us (541us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762884.040978:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 521 00000100:00100000:1.0:1789762884.040995:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+2:2394:x1876702067462656:12345-192.168.203.37@tcp:9:kworker.0 Request processed in 571us (818us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762884.041008:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 175 00010000:00010000:3.0:1789762884.041112:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762884.224050:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762885.120153:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1789762885.120197:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074708864:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762885.120244:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074708992:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762885.120332:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762885.120399:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074708992 00000100:00100000:0.0:1789762885.120410:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074708992 00000100:00100000:3.0:1789762885.120421:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762885.120457:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074709504:0@lo:400:kworker.0 00000100:00100000:3.0:1789762885.120477:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1ab8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074708608:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762885.120491:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074708864 00000100:00100000:0.0:1789762885.120495:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074708864 00000100:00100000:2.0:1789762885.120504:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762885.120546:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762885.120547:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1789762885.120548:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074710016:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.120582:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:1.0:1789762885.120597:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074708992:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762885.120621:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074708736:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762885.120622:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074708864:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762885.120631:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:1.0:1789762885.120654:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00002000:00000024:0.0:1789762885.120670:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762885.120693:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:2.0:1789762885.120707:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074709120:0@lo:400:kworker.0 00000100:00100000:3.0:1789762885.120717:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762885.120784:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074709376:0@lo:400:kworker.0 00000100:00100000:1.0:1789762885.120827:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074708992:12345-0@lo:13:osp-pre-1-1.0 Request processed in 235us (483us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762885.120827:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762885.120844:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074708864:12345-0@lo:13:osp-pre-1-0.0 Request processed in 236us (420us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762885.120862:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:3.0:1789762885.120879:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762885.120896:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074709632:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.120898:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00100000:1.0:1789762885.120919:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074708608 00000100:00100000:1.0:1789762885.120926:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074708608 00000100:00100000:3.0:1789762885.120969:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074709888:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.120981:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074708736 00000100:00100000:1.0:1789762885.120989:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:0.0:1789762885.120991:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074708736 00000100:00100000:2.0:1789762885.120992:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762885.121004:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074708608:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762885.121014:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:1.0:1789762885.121039:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762885.121044:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074710144:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.121046:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:3.0:1789762885.121056:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7fbad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074710400:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.121068:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074708736:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789762885.121086:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762885.121115:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762885.121116:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074708608:12345-0@lo:13:osp-pre-0-0.0 Request processed in 108us (576us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762885.121123:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762885.121139:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00100000:0.0:1789762885.121172:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074708736:12345-0@lo:13:osp-pre-0-1.0 Request processed in 105us (451us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762885.121184:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:1.0:1789762885.121197:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074709376 00000100:00100000:1.0:1789762885.121204:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074709376 00000100:00100000:3.0:1789762885.121228:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074708864:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762885.121253:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074708992:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762885.121262:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074709248:0@lo:400:kworker.0 00000100:00100000:1.0:1789762885.121268:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074709632 00000100:00100000:3.0:1789762885.121271:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1ab8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074708608:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762885.121277:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074709632 00000100:00100000:3.0:1789762885.121322:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074708736:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762885.121330:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762885.121336:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 345 00000100:00100000:2.0:1789762885.121342:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074709120 00000100:00100000:1.0:1789762885.121348:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074709376:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762885.121349:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074709120 00000100:00100000:0.0:1789762885.121375:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074709760:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.121422:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762885.121435:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:1.0:1789762885.121436:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074709376:12345-0@lo:400:kworker.0 Request processed in 90us (584us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762885.121457:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074710272:0@lo:400:kworker.0 00000100:00100000:3.0:1789762885.121457:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074710400 00000100:00100000:2.0:1789762885.121464:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1553850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074709120:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762885.121467:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 345 00000100:00100000:3.0:1789762885.121475:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074710400 00000100:00100000:0.0:1789762885.121508:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762885.121511:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 346 00000100:00100000:1.0:1789762885.121532:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074709632:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762885.121558:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074710272 00000100:00100000:2.0:1789762885.121563:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1553850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074709120:12345-0@lo:400:kworker.0 Request processed in 111us (746us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762885.121565:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074710272 00000100:00100000:2.0:1789762885.121578:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000100:00100000:0.0:1789762885.121582:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074709504 00000100:00100000:0.0:1789762885.121591:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074709504 00000100:00100000:1.0:1789762885.121604:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074709632:12345-0@lo:400:kworker.0 Request processed in 81us (620us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762885.121617:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 346 00000100:00100000:3.0:1789762885.121630:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:0.0:1789762885.121640:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074710016 00000100:00100000:0.0:1789762885.121645:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074710016 00000100:00100000:3.0:1789762885.121651:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074710400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762885.121663:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074709888 00000100:00100000:2.0:1789762885.121669:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074709888 00000100:00100000:0.0:1789762885.121679:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074710144 00000100:00100000:0.0:1789762885.121685:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074710144 00000100:00100000:1.0:1789762885.121690:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074709376:0@lo:400:kworker.0 00000100:00100000:2.0:1789762885.121724:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074709248 00000100:00100000:2.0:1789762885.121729:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074709248 00000100:00100000:3.0:1789762885.121738:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074710400:12345-0@lo:400:kworker.0 Request processed in 87us (618us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762885.121739:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074709760 00000100:00100000:1.0:1789762885.121744:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7fbad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074710400:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.121748:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074709760 00000100:00100000:2.0:1789762885.121762:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 522 00000100:00100000:3.0:1789762885.121763:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:2.0:1789762885.121779:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074709504:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.121793:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 523 00000100:00100000:0.0:1789762885.121809:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074710016:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762885.121817:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:3.0:1789762885.121839:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074710272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762885.121840:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074709504:12345-0@lo:400:kworker.0 Request processed in 66us (1332us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762885.121854:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 524 00000100:00100000:0.0:1789762885.121863:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074710016:12345-0@lo:400:kworker.0 Request processed in 56us (1252us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762885.121885:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 522 00000100:00100000:1.0:1789762885.121886:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074709888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.121904:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 523 00000100:00100000:2.0:1789762885.121961:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 525 00000100:00100000:3.0:1789762885.121963:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074710272:12345-0@lo:400:kworker.0 Request processed in 128us (452us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762885.121976:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 526 00000100:00100000:3.0:1789762885.121979:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1789762885.121983:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074710144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762885.121990:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074709888:12345-0@lo:400:kworker.0 Request processed in 112us (974us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762885.122004:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074709248:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762885.122010:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 524 00000100:00100000:1.0:1789762885.122057:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 527 00000100:00100000:1.0:1789762885.122074:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074709760:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762885.122082:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074710144:12345-0@lo:400:kworker.0 Request processed in 109us (979us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762885.122086:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074709248:12345-0@lo:400:kworker.0 Request processed in 84us (759us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762885.122096:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762885.122103:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 525 00000100:00080000:3.0:1789762885.122108:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762885.122116:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 526 00000100:00080000:3.0:1789762885.122121:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789762885.122128:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762885.122136:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074709760:12345-0@lo:400:kworker.0 Request processed in 66us (714us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762885.122149:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 527 00000100:00100000:2.0:1789762885.122186:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074709120:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.122190:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074709248:0@lo:400:kworker.0 00000100:00100000:1.0:1789762885.122236:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074709888:0@lo:400:kworker.0 00000100:00100000:2.0:1789762885.122236:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074709632:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.122238:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074709760:0@lo:400:kworker.0 00000100:00100000:2.0:1789762885.122283:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074710144:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.122294:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074710272:0@lo:400:kworker.0 00000100:00100000:2.0:1789762885.122303:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074709504:0@lo:400:kworker.0 00000100:00100000:2.0:1789762885.122332:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074710016:0@lo:400:kworker.0 00010000:00010000:1.0:1789762885.248074:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762886.272068:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762887.168097:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7fbad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074710528:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762887.168176:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762887.168233:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074710528 00000100:00100000:1.0:1789762887.168254:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074710528 00000100:00100000:1.0:1789762887.168305:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 347 00000100:00100000:1.0:1789762887.168316:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074710528:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762887.168356:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762887.168411:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074710528:12345-0@lo:41:osp-pre-1-0.0 Request processed in 96us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762887.168420:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 347 00000100:00100000:3.0:1789762887.168454:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7fbad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074710528:0@lo:41:osp-pre-1-0.0 00000001:00100000:0.0:1789762887.175683:0:9000:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000020:00000080:0.0:1789762887.175688:0:9000:0:(genops.c:885:class_export_put()) final put ffff9c37d1024800/fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000020:00000080:1.0:1789762887.175711:0:38:0:(genops.c:844:class_export_destroy()) destroying export ffff9c37d1024800/fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 for lustre-OST0001 00000020:00000080:1.0:1789762887.175749:0:38:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9c37d1023800 for lustre-OST0001 00010000:00010000:3.0:1789762887.296050:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762888.192147:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074710656:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789762888.192247:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762888.192286:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074710656 00000100:00100000:3.0:1789762888.192294:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074710656 00000100:00100000:3.0:1789762888.192343:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 348 00000100:00100000:3.0:1789762888.192352:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074710656:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789762888.192385:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789762888.192430:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074710656:12345-0@lo:41:osp-pre-0-1.0 Request processed in 80us (186us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762888.192436:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 348 00000100:00100000:0.0:1789762888.192472:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074710656:0@lo:41:osp-pre-0-1.0 00010000:00010000:3.0:1789762888.320044:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762889.160344:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762889.160421:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762889.160429:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067463040 00000100:00100000:0.0:1789762889.160443:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067463040 00000100:00100000:1.0:1789762889.160464:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762889.160511:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067463168 00000100:00100000:3.0:1789762889.160528:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067463168 00000100:00100000:0.0:1789762889.160536:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 367 00000100:00100000:1.0:1789762889.160537:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762889.160546:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067463424 00000100:00100000:2.0:1789762889.160555:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067463424 00000100:00100000:0.0:1789762889.160560:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1553450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067463040:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762889.160619:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762889.160652:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 215 00000100:00100000:0.0:1789762889.160668:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1553450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067463040:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 114us (342us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762889.160670:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067463424:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762889.160678:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 367 00000100:00100000:3.0:1789762889.160685:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067463296 00000100:00100000:2.0:1789762889.160720:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067463424:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 54us (253us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762889.160724:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067463296 00000100:00100000:2.0:1789762889.160733:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 215 00000100:00080000:2.0:1789762889.160782:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762889.160793:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762889.160802:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762889.160809:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067463680 00000100:00100000:3.0:1789762889.160815:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067463680 00000100:00100000:0.0:1789762889.160826:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 528 00000100:00100000:2.0:1789762889.160841:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 529 00000100:00100000:0.0:1789762889.160846:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067463168:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762889.160861:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067463296:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762889.160976:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067463168:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 79us (504us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762889.160979:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067463296:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 67us (392us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762889.161009:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762889.161058:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 530 00000100:00100000:2.0:1789762889.161073:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 529 00000100:00100000:3.0:1789762889.161083:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067463680:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762889.161099:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 528 00000100:00100000:2.0:1789762889.161112:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067463552 00000100:00100000:2.0:1789762889.161120:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067463552 00000100:00100000:2.0:1789762889.161167:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 531 00000100:00100000:2.0:1789762889.161190:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067463552:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:0.0:1789762889.161205:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762889.161209:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762889.161253:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067463552:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 76us (246us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762889.161260:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 531 00000100:00100000:3.0:1789762889.161568:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067463680:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 201us (663us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762889.161587:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 530 00010000:00010000:1.0:1789762889.344197:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762890.240128:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d204c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074710784:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762890.240228:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762890.240317:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1abad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074711168:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762890.240343:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074711040:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762890.240378:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762890.240381:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d2a10000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074711296:0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.240422:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074710784 00000100:00100000:0.0:1789762890.240432:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074710784 00000100:00100000:2.0:1789762890.240440:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d1aba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074711552:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762890.240474:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074711168 00000100:00100000:1.0:1789762890.240486:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074711168 00000100:00100000:3.0:1789762890.240525:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762890.240526:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:2.0:1789762890.240527:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762890.240554:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074710784:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789762890.240595:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762890.240626:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074711552 00000100:00100000:3.0:1789762890.240633:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d2a13800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074711680:0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.240639:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074711552 00000100:00100000:2.0:1789762890.240648:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d2a13480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074712192:0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.240676:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074710784:12345-0@lo:13:osp-pre-1-1.0 Request processed in 123us (450us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762890.240690:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:1.0:1789762890.240703:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:3.0:1789762890.240715:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762890.240732:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074711168:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762890.240746:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:0.0:1789762890.240770:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca559f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074711552:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762890.240784:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c9df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074710912:0@lo:400:kworker.0 00000100:00100000:3.0:1789762890.240789:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d2a12680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074712064:0@lo:400:kworker.0 00002000:00000024:1.0:1789762890.240799:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3859808256 avail 3857711104 00002000:00000024:0.0:1789762890.240800:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762890.240858:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762890.240859:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762890.240957:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca559f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074711552:12345-0@lo:13:osp-pre-1-0.0 Request processed in 155us (404us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762890.240970:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00100000:2.0:1789762890.241003:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d2a10700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074711424:0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.241014:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074711168:12345-0@lo:13:osp-pre-0-1.0 Request processed in 287us (639us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762890.241020:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074711296 00000100:00100000:3.0:1789762890.241021:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074712576:0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.241042:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074711296 00000100:00100000:1.0:1789762890.241049:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00000100:00100000:2.0:1789762890.241066:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762890.241099:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762890.241118:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d2a10a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074711936:0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.241150:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074711680 00000100:00100000:1.0:1789762890.241158:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074711680 00000100:00100000:0.0:1789762890.241159:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074712064 00000100:00100000:0.0:1789762890.241186:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074712064 00000100:00100000:2.0:1789762890.241193:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762890.241236:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074712448:0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.241276:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074711936 00000100:00100000:3.0:1789762890.241287:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074710912 00000100:00100000:0.0:1789762890.241288:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074711936 00000100:00100000:1.0:1789762890.241288:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 532 00000100:00100000:2.0:1789762890.241296:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762890.241306:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074710912 00000100:00100000:1.0:1789762890.241320:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074711296:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762890.241334:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789762890.241349:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074712320:0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.241368:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 533 00000100:00100000:0.0:1789762890.241388:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074712064:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762890.241402:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 368 00000100:00100000:3.0:1789762890.241426:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1551850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074710912:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762890.241442:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762890.241477:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074711296:12345-0@lo:400:kworker.0 Request processed in 161us (960us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762890.241491:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074712064:12345-0@lo:400:kworker.0 Request processed in 110us (642us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762890.241492:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 532 00000100:00100000:2.0:1789762890.241509:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074711424 00000100:00100000:0.0:1789762890.241514:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 533 00000100:00100000:2.0:1789762890.241516:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074711424 00000100:00100000:1.0:1789762890.241558:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074712320 00000100:00100000:3.0:1789762890.241566:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1551850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074710912:12345-0@lo:400:kworker.0 Request processed in 142us (714us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762890.241569:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074712320 00000100:00100000:0.0:1789762890.241570:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 534 00000100:00100000:2.0:1789762890.241589:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 349 00000100:00100000:0.0:1789762890.241598:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074711680:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762890.241609:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 368 00000100:00100000:2.0:1789762890.241612:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074711424:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.241634:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 535 00000100:00100000:1.0:1789762890.241656:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee760a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074711936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.241696:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074711680:12345-0@lo:400:kworker.0 Request processed in 104us (989us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762890.241709:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 534 00000100:00100000:3.0:1789762890.241726:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074712576 00000100:00100000:3.0:1789762890.241741:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074712576 00000100:00100000:2.0:1789762890.241746:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074711424:12345-0@lo:400:kworker.0 Request processed in 135us (688us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762890.241759:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 536 00000100:00100000:1.0:1789762890.241763:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee760a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074711936:12345-0@lo:400:kworker.0 Request processed in 107us (560us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762890.241764:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 349 00000100:00100000:0.0:1789762890.241781:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee762a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074712320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.241804:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 535 00000100:00100000:2.0:1789762890.241821:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762890.241833:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074712448 00000100:00100000:0.0:1789762890.241851:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee762a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074712320:12345-0@lo:400:kworker.0 Request processed in 77us (417us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762890.241864:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074712448 00000100:00100000:0.0:1789762890.241866:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 536 00000100:00100000:1.0:1789762890.241868:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074712192 00000100:00100000:2.0:1789762890.241869:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074712704:0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.241883:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074712192 00000100:00080000:0.0:1789762890.241906:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762890.241914:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762890.241944:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762890.241949:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762890.241970:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762890.241978:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762890.241999:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074712704 00000100:00100000:3.0:1789762890.242010:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074712704 00000100:00100000:1.0:1789762890.242027:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 537 00000100:00100000:0.0:1789762890.242047:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d2a10e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074711808:0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.242049:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee762680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074712192:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762890.242069:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:2.0:1789762890.242072:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1abad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074711168:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762890.242099:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074712576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762890.242118:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d1aba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074711552:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762890.242119:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762890.242200:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee762680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074712192:12345-0@lo:400:kworker.0 Request processed in 137us (364us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762890.242215:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 537 00000100:00100000:2.0:1789762890.242217:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d2a13480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074712192:0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.242224:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d204c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074710784:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762890.242269:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074712576:12345-0@lo:400:kworker.0 Request processed in 177us (1168us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762890.242286:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074711808 00000100:00100000:3.0:1789762890.242291:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00100000:1.0:1789762890.242294:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074711808 00000100:00100000:0.0:1789762890.242317:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074711040 00000100:00100000:0.0:1789762890.242326:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074711040 00000100:00100000:3.0:1789762890.242347:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:3.0:1789762890.242374:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074712448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762890.242376:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:0.0:1789762890.242398:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:1.0:1789762890.242401:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 350 00000100:00100000:2.0:1789762890.242411:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074712704:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.242413:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074711040:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762890.242428:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074711808:12345-0@lo:400:kworker.0 00002000:00000024:0.0:1789762890.242443:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762890.242472:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074712448:12345-0@lo:400:kworker.0 Request processed in 107us (1181us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762890.242485:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:2.0:1789762890.242536:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074712704:12345-0@lo:400:kworker.0 Request processed in 128us (565us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762890.242560:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074711808:12345-0@lo:400:kworker.0 Request processed in 137us (438us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762890.242563:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:0.0:1789762890.242573:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074711040:12345-0@lo:13:osp-pre-0-0.0 Request processed in 156us (598us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762890.242593:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074712320:0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.242600:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 350 00000100:00100000:0.0:1789762890.242605:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000100:00100000:3.0:1789762890.242662:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c9df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074710912:0@lo:400:kworker.0 00000100:00100000:2.0:1789762890.242689:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074712576:0@lo:400:kworker.0 00000100:00100000:3.0:1789762890.242710:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d2a10700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074711424:0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.242750:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074711040:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762890.242755:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d2a10a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074711936:0@lo:400:kworker.0 00000100:00100000:2.0:1789762890.242757:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d2a10000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074711296:0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.242786:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d2a10e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074711808:0@lo:400:kworker.0 00000100:00100000:3.0:1789762890.242807:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074712448:0@lo:400:kworker.0 00000100:00100000:2.0:1789762890.242816:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d2a13800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074711680:0@lo:400:kworker.0 00000100:00100000:2.0:1789762890.242852:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d2a12680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074712064:0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.242971:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074712704:0@lo:400:kworker.0 00010000:00010000:1.0:1789762890.368052:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762891.393023:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762891.867738:0:17162:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:0.0:1789762891.867748:0:17162:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:0.0:1789762891.867757:0:17162:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:0.0:1789762891.867763:0:17162:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:0.0:1789762891.867767:0:17162:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:0.0:1789762891.867773:0:17162:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37ff763f00 rate 20 gen 0 00000100:00100000:0.0:1789762891.867776:0:17162:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:0.0:1789762891.867779:0:17162:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:0.0:1789762891.867783:0:17162:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:0.0:1789762891.867787:0:17162:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37ff763200 rate 20 gen 0 00000100:00100000:0.0:1789762891.867880:0:17162:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:0.0:1789762891.867884:0:17162:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:0.0:1789762891.867889:0:17162:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:0.0:1789762891.867893:0:17162:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:0.0:1789762891.867895:0:17162:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:0.0:1789762891.867898:0:17162:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37ff763800 rate 10 gen 0 00000100:00100000:0.0:1789762891.867900:0:17162:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:0.0:1789762891.867903:0:17162:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:0.0:1789762891.867907:0:17162:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:0.0:1789762891.867911:0:17162:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37ff763d00 rate 10 gen 0 00000100:00100000:0.0:1789762891.879591:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762891.879642:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067463808 00000100:00100000:2.0:1789762891.879654:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067463808 00000100:00080000:0.0:1789762891.879693:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762891.879727:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 293 00000100:00100000:2.0:1789762891.879742:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:15109:x1876702067463808:12345-192.168.203.37@tcp:101:mkdir.0 00010000:00010000:2.0:1789762891.879761:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762891.879783:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176891 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c912 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.879844:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f176898 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.879872:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f176898 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.879887:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f176898 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762891.879949:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762891.879954:0:6420:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762891.879975:0:6420:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f176898 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.880004:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769400/0xdf5a7ac59f176891 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c912 expref: 7 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.880024:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f176898 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c912 expref: 6 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.880034:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762891.880038:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762891.880041:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769a00, rc 0) 00000100:00100000:2.0:1789762891.880103:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15109:x1876702067463808:12345-192.168.203.37@tcp:101:mkdir.0 Request processed in 365us (515us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762891.880113:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 293 00000100:00100000:0.0:1789762891.881263:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762891.881305:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067464064 00000100:00100000:2.0:1789762891.881312:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067464064 00000100:00080000:0.0:1789762891.881348:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762891.881359:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 294 00000100:00100000:2.0:1789762891.881371:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:15109:x1876702067464064:12345-192.168.203.37@tcp:36:mkdir.0 00000100:00100000:2.0:1789762891.881451:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee761500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074712832:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762891.881506:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762891.881538:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074712832 00000100:00100000:3.0:1789762891.881544:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074712832 00000100:00100000:2.0:1789762891.881561:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252f780 going to sleep for 11 seconds 00000100:00100000:3.0:1789762891.881584:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 351 00000100:00100000:3.0:1789762891.881593:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074712832:12345-0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.881652:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074712832:12345-0@lo:1000:mdt00_005.0 Request processed in 61us (147us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762891.881666:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 351 00000100:00100000:2.0:1789762891.881691:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee761500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074712832:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762891.881747:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee761500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074712960:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762891.881780:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762891.881804:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252f780 going to sleep for 11 seconds 00000100:00100000:3.0:1789762891.881818:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074712960 00000100:00100000:3.0:1789762891.881823:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074712960 00000100:00100000:3.0:1789762891.881848:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:3.0:1789762891.881855:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074712960:12345-0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.882069:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074712960:12345-0@lo:1000:mdt00_005.0 Request processed in 214us (290us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789762891.882075:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00100000:2.0:1789762891.882102:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee761500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074712960:0@lo:1000:mdt00_005.0 00010000:00010000:2.0:1789762891.882150:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.882176:0:13389:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.882186:0:13389:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762891.882200:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee761500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074713088:0@lo:101:mdt00_005.0 00000100:00100000:2.0:1789762891.882232:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762891.882269:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074713088 00000100:00100000:2.0:1789762891.882270:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252f840 going to sleep for 11 seconds 00000100:00100000:0.0:1789762891.882275:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074713088 00000100:00100000:0.0:1789762891.882313:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 353 00000100:00100000:0.0:1789762891.882322:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074713088:12345-0@lo:101:mdt00_005.0 00010000:00010000:0.0:1789762891.882331:0:13302:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762891.882342:0:13302:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f1768a6 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f17689f expref: -99 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.882359:0:13302:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f176898 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c912 expref: 5 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.882370:0:13302:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f1768a6 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f17689f expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.882402:0:13302:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f176898 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c912 expref: 5 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.882418:0:13302:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f176898 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c912 expref: 5 pid: 6420 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762891.882429:0:13302:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac4000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:13302:1876702074713216:192.168.203.37@tcp:104: 00000100:00100000:0.0:1789762891.882479:0:13302:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5b1c300 going to sleep for 11 seconds 00000100:00100000:2.0:1789762891.883294:0:13302:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac4000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:13302:1876702074713216:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762891.883312:0:13302:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f1768a6 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17689f expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.883322:0:13302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762891.883325:0:13302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762891.883329:0:13302:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f1768a6 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17689f expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.883351:0:13302:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca315800, rc 0) 00000100:00100000:2.0:1789762891.883394:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:13389:x1876702074713088:12345-0@lo:101:mdt00_005.0 Request processed in 1073us (1160us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762891.883402:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 353 00000100:00100000:3.0:1789762891.883475:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee761500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074713088:0@lo:101:mdt00_005.0 00010000:00010000:3.0:1789762891.883488:0:13389:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.883497:0:13389:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762891.883514:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762891.883544:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067464192 00000100:00100000:3.0:1789762891.883550:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067464192 00010000:00010000:3.0:1789762891.883612:0:9878:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f176898 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c912 expref: 6 pid: 6420 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762891.883637:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 81 00000100:00100000:3.0:1789762891.883645:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067464192:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762891.883655:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762891.883660:0:9878:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f176898 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c912 expref: 6 pid: 6420 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.883679:0:9878:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f176898 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c912 expref: 6 pid: 6420 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.883691:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f176898 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c912 expref: 6 pid: 6420 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.883700:0:9878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762891.883703:0:9878:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f1768a6 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17689f expref: 6 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.883710:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f1768a6 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f17689f expref: 6 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.883717:0:9878:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f1768a6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f17689f expref: 6 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.883734:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f1768a6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17689f expref: 6 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762891.883741:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccdc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:9878:1876702074713344:0@lo:105: 00000100:00100000:3.0:1789762891.883768:0:9878:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762891.883807:0:8068:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074713344 00000100:00100000:3.0:1789762891.883807:0:9878:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c717f780 going to sleep for 11 seconds 00000100:00100000:0.0:1789762891.883813:0:8068:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074713344 00000100:00100000:0.0:1789762891.883845:0:8068:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 53 00000100:00100000:0.0:1789762891.883852:0:8068:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:9878:x1876702074713344:12345-0@lo:105: 00010000:00010000:0.0:1789762891.883861:0:8068:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.883878:0:8068:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.883884:0:8068:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.883889:0:8068:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.883895:0:8068:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.883901:0:8068:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.883945:0:13389:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.883948:0:8068:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c37d1769200) 00010000:00010000:3.0:1789762891.883956:0:13389:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762891.883981:0:13389:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1789762891.883995:0:8068:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:9878:x1876702074713344:12345-0@lo:105: Request processed in 144us (226us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762891.884003:0:8068:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 53 00010000:00010000:3.0:1789762891.884005:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.884011:0:13389:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.884017:0:13389:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762891.884028:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074713472:0@lo:101:mdt00_005.0 00000100:00100000:3.0:1789762891.884057:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762891.884075:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d196bb40 going to sleep for 11 seconds 00000100:00100000:0.0:1789762891.884087:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074713472 00000100:00100000:0.0:1789762891.884092:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074713472 00000100:00100000:3.0:1789762891.884098:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccdc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:9878:1876702074713344:0@lo:105: 00010000:00010000:3.0:1789762891.884114:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762891.884126:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:0.0:1789762891.884135:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:13389:x1876702074713472:12345-0@lo:101:mdt00_005.0 00010000:00010000:0.0:1789762891.884143:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1789762891.884144:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067464192:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 499us (629us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762891.884152:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 81 00010000:00010000:0.0:1789762891.884171:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f1768b4 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f1768ad expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.884199:0:6424:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f1768b4 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f1768ad expref: 8 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.884209:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f1768b4 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1768ad expref: 8 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.884215:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37e37a7f00) 00010000:00010000:0.0:1789762891.884217:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762891.884220:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca314000, rc 0) 00000100:00100000:0.0:1789762891.884258:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074713472:12345-0@lo:101:mdt00_005.0 Request processed in 126us (202us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762891.884265:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00100000:3.0:1789762891.884281:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074713472:0@lo:101:mdt00_005.0 00010000:00010000:3.0:1789762891.884377:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1768b4 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.884387:0:13389:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1768b4 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762891.884398:0:13389:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1789762891.884419:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074713600:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.884448:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762891.884466:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d196bd80 going to sleep for 11 seconds 00000100:00100000:2.0:1789762891.884489:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074713600 00000100:00100000:2.0:1789762891.884496:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074713600 00000100:00100000:2.0:1789762891.884531:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:2.0:1789762891.884542:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074713600:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762891.884603:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074713600:12345-0@lo:1000:mdt00_005.0 Request processed in 64us (152us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762891.884612:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00100000:3.0:1789762891.884625:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074713600:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.884657:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074713728:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.884680:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762891.884700:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d196bb40 going to sleep for 11 seconds 00000100:00100000:2.0:1789762891.884716:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074713728 00000100:00100000:2.0:1789762891.884722:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074713728 00000100:00100000:2.0:1789762891.884754:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:2.0:1789762891.884763:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074713728:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762891.884972:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074713728:12345-0@lo:1000:mdt00_005.0 Request processed in 208us (289us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789762891.884977:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074713728:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762891.884991:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:3.0:1789762891.885047:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074713856:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.885070:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762891.885080:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d196bd80 going to sleep for 11 seconds 00000100:00100000:0.0:1789762891.885101:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074713856 00000100:00100000:0.0:1789762891.885106:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074713856 00000100:00100000:0.0:1789762891.885131:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 357 00000100:00100000:0.0:1789762891.885141:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074713856:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762891.885298:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074713856:12345-0@lo:1000:mdt00_005.0 Request processed in 160us (227us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789762891.885304:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 357 00000100:00100000:3.0:1789762891.885329:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074713856:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.885376:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074713984:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.885402:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762891.885418:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d196bd80 going to sleep for 11 seconds 00000100:00100000:0.0:1789762891.885434:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074713984 00000100:00100000:0.0:1789762891.885439:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074713984 00000100:00100000:0.0:1789762891.885464:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 358 00000100:00100000:0.0:1789762891.885470:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff617100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074713984:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1789762891.885523:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff617100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074713984:12345-0@lo:1000:mdt00_005.0 Request processed in 54us (121us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762891.885530:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 358 00000100:00100000:3.0:1789762891.885538:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffd880 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074713984:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.885571:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074714112:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.885589:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762891.885603:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d041d600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762891.885622:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074714112 00000100:00100000:2.0:1789762891.885628:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074714112 00000100:00100000:2.0:1789762891.885663:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 359 00000100:00100000:2.0:1789762891.885673:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074714112:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762891.885737:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074714112:12345-0@lo:1000:mdt00_005.0 Request processed in 66us (145us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762891.885746:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 359 00000100:00100000:3.0:1789762891.885756:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074714112:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.885806:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074714240:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.885837:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762891.885853:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d041d600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762891.885862:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074714240 00000100:00100000:2.0:1789762891.885868:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074714240 00000100:00100000:2.0:1789762891.885900:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1789762891.885911:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074714240:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762891.886007:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:13389:x1876702074714240:12345-0@lo:1000:mdt00_005.0 Request processed in 97us (166us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762891.886013:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074714240:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762891.886018:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000040:00080000:3.0:1789762891.887349:0:13389:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].11, 984 off40792 00000040:00080000:3.0:1789762891.887500:0:13389:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].11, 1136 off42104 00000004:00080000:3.0:1789762891.887506:0:13389:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37d041d000 with version 24 00000004:00080000:0.0:1789762891.887535:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 24 rpc_version 24 00000020:00100000:0.0:1789762891.887554:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762891.887560:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c36cbac6d80 x1876702074714496/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2416/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762891.887573:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac6d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074714496:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762891.887608:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762891.887628:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5b1c3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762891.887652:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074714496 00000100:00100000:2.0:1789762891.887658:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074714496 00000100:00100000:2.0:1789762891.887691:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 361 00000100:00100000:2.0:1789762891.887700:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7224:x1876702074714496:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762891.888473:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:7224:x1876702074714496:12345-0@lo:1000:osp_up0-1.0 Request processed in 771us (862us total) trans 4294967425 rc 0/0 00000100:00100000:2.0:1789762891.888483:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 361 00000100:00100000:0.0:1789762891.888516:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac6d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074714496:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1789762891.888567:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1768bb lrc: 3/0,1 mode: --/PW res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.888577:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1768bb lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.888586:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1768bb lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762891.888597:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d37c763f reply state 00000000ef73cdc6 transno 4294967423 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1768bb lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.888604:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1768bb lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.888609:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762891.888611:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1789762891.888613:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762891.888617:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14:0x0]: returning size 16384 00000004:00080000:3.0:1789762891.888621:0:13389:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1789762891.888629:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.888635:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.888641:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.888667:0:13389:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1789762891.888671:0:13389:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1789762891.888676:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1768b4 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.888682:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762891.888682:0:13389:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f1768b4 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.888692:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.888700:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769200 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:0.0:1789762891.888705:0:6407:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762891.888712:0:6407:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762891.888732:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15109:x1876702067464064:12345-192.168.203.37@tcp:36:mkdir.0 Request processed in 7364us (7469us total) trans 4294967423 rc 0/0 00010000:00010000:0.0:1789762891.888733:0:6407:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762891.888742:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 294 00010000:00010000:0.0:1789762891.888744:0:6407:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1789762891.888769:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.888775:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d1769200/0xdf5a7ac59f17689f lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f1768a6 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762891.888791:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789762891.888808:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff699f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074714624:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762891.888809:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762891.888812:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762891.888816:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762891.888852:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762891.888943:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074714624 00000100:00100000:0.0:1789762891.888947:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074714624 00000100:00100000:0.0:1789762891.888990:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 56 00000100:00100000:0.0:1789762891.888999:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c91c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3425:x1876702074714624:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762891.889017:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762891.889026:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f1768a6 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f17689f expref: 9 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.889037:0:6406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762891.889040:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762891.889075:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c91c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+8:3425:x1876702074714624:12345-0@lo:103:ldlm_bl.0 Request processed in 78us (223us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762891.889082:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 56 00000100:00100000:3.0:1789762891.889098:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff699f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074714624:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762891.894219:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762891.894262:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067464320 00000100:00100000:2.0:1789762891.894268:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067464320 00000100:00100000:2.0:1789762891.894307:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 295 00000100:00100000:2.0:1789762891.894318:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:15110:x1876702067464320:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762891.894329:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762891.894339:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f1768c2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c919 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.894380:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f1768c9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.894392:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f1768c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.894402:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f1768c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762891.894429:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762891.894433:0:11692:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762891.894447:0:11692:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f1768c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.894466:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1769c00/0xdf5a7ac59f1768c2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c919 expref: 7 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.894482:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f1768c9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c919 expref: 6 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.894490:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762891.894493:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762891.894496:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768c00, rc 0) 00000100:00100000:2.0:1789762891.894535:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15110:x1876702067464320:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 218us (317us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762891.894543:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 295 00000100:00100000:0.0:1789762891.895599:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762891.895628:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067464576 00000100:00100000:3.0:1789762891.895634:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067464576 00000100:00100000:3.0:1789762891.895664:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 296 00000100:00100000:3.0:1789762891.895672:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15110:x1876702067464576:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:3.0:1789762891.895681:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762891.895683:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1789762891.895694:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768c00/0xdf5a7ac59f1768c9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c919 expref: 6 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.895706:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762891.895716:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762891.895724:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f1768d0 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c920 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.895794:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f1768d7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.895803:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f1768d7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.895809:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f1768d7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.895819:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.895825:0:6419:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f1768b4 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f1768ad expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.895831:0:6419:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.895849:0:6419:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f1768b4 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xdf5a7ac59f1768ad expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762891.895857:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6419:1876702074714752:0@lo:104: 00000100:00100000:3.0:1789762891.895883:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762891.895940:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d196bd80 going to sleep for 11 seconds 00000100:00100000:2.0:1789762891.896010:0:6403:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074714752 00000100:00100000:2.0:1789762891.896017:0:6403:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074714752 00000100:00100000:2.0:1789762891.896055:0:6403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 54 00000100:00100000:2.0:1789762891.896067:0:6403:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6419:x1876702074714752:12345-0@lo:104: 00010000:00010000:2.0:1789762891.896078:0:6403:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f1768b4 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.896143:0:6403:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1789762891.896152:0:6403:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6419:x1876702074714752:12345-0@lo:104: Request processed in 87us (265us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762891.896152:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6419:1876702074714752:0@lo:104: 00010000:00010000:3.0:1789762891.896191:0:6419:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762891.896197:0:6403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 54 00010000:00010000:3.0:1789762891.896229:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762891.896233:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f1768b4 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.896240:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933600 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:3.0:1789762891.896244:0:8069:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xdf5a7ac59f1768b4 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762891.896249:0:8069:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xdf5a7ac59f1768b4 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.896267:0:8069:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f1768b4 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.896277:0:8069:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1789762891.896301:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f1768b4 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762891.896308:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074714880:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789762891.896315:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37c5933600/0xdf5a7ac59f1768ad lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f1768b4 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.896336:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762891.896338:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762891.896340:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762891.896349:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762891.896377:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074714880 00000100:00100000:1.0:1789762891.896384:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074714880 00010000:00010000:1.0:1789762891.896402:0:6405:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f1768b4 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xdf5a7ac59f1768ad expref: 8 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762891.896427:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 82 00000100:00100000:1.0:1789762891.896434:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3425:x1876702074714880:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762891.896442:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1789762891.907030:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967425 is committed 00010000:00010000:2.0:1789762891.908797:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f1768b4 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xdf5a7ac59f1768ad expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.908814:0:6405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37e37a7f00) 00010000:00010000:2.0:1789762891.908818:0:6405:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.908835:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.908841:0:6405:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.908859:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762891.908904:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074714880:12345-0@lo:103:ldlm_bl.0 Request processed in 12470us (12555us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762891.908911:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 82 00000100:00100000:2.0:1789762891.908988:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074714880:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789762891.909205:0:6419:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.909222:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762891.909415:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6419:1876702074715008:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762891.909449:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762891.909476:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d25693c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762891.909487:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074715008 00000100:00100000:2.0:1789762891.909492:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074715008 00000100:00100000:2.0:1789762891.909525:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 362 00000100:00100000:2.0:1789762891.909534:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6419:x1876702074715008:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789762891.909603:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6419:x1876702074715008:12345-0@lo:1000:mdt00_001.0 Request processed in 72us (153us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762891.909616:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 362 00000100:00100000:0.0:1789762891.909632:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6419:1876702074715008:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1789762891.909670:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315e00/0xdf5a7ac59f1768e5 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909679:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315e00/0xdf5a7ac59f1768e5 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909686:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315e00/0xdf5a7ac59f1768e5 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909705:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f1768d7 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909710:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f1768d7 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909714:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f1768d7 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909719:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f1768d7 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909723:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762891.909726:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f1768d7 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909731:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f1768d7 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909739:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762891.909741:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762891.909744:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f1768d7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909748:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f1768d7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909760:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909764:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909774:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909778:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762891.909781:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909785:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909793:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37e37a7f00) 00010000:00010000:0.0:1789762891.909795:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762891.909797:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909801:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f1768de lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909846:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f1768d0 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c920 expref: 7 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909860:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315e00/0xdf5a7ac59f1768e5 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c920 expref: 6 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.909864:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37e37a7200) 00010000:00010000:0.0:1789762891.909866:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762891.909868:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca315e00, rc 0) 00000100:00100000:0.0:1789762891.909913:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15110:x1876702067464576:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 14232us (14306us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762891.909961:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 296 00000100:00100000:2.0:1789762891.911121:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762891.912430:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067464832 00000100:00100000:3.0:1789762891.912435:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067464832 00000100:00100000:3.0:1789762891.912472:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 297 00000100:00100000:3.0:1789762891.912484:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15110:x1876702067464832:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:3.0:1789762891.912511:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762891.912526:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933600/0xdf5a7ac59f1768ec lrc: 2/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c927 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762891.912583:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14:0x0]: returning size 16384 00010000:00010000:3.0:1789762891.912598:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933600/0xdf5a7ac59f1768ec lrc: 1/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c927 expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.912618:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762891.912620:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762891.912625:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933600/0xdf5a7ac59f1768ec lrc: 0/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c927 expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.912638:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5933600, rc 0) 00000100:00100000:3.0:1789762891.912679:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15110:x1876702067464832:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 198us (1565us total) trans 0 rc 301/301 00000100:00100000:3.0:1789762891.912694:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 297 00000100:00100000:2.0:1789762891.913873:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762891.913977:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067464960 00000100:00100000:2.0:1789762891.913981:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067464960 00000100:00100000:2.0:1789762891.914009:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 298 00000100:00100000:2.0:1789762891.914016:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a12680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15110:x1876702067464960:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762891.914024:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762891.914034:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f1768f3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c92e expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.914063:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f1768fa lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.914071:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f1768fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.914077:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f1768fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762891.914091:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762891.914094:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762891.914104:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f1768fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.914120:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768600/0xdf5a7ac59f1768f3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c92e expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.914134:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f1768fa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c92e expref: 7 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.914146:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762891.914149:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762891.914151:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768400, rc 0) 00000100:00100000:2.0:1789762891.914202:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a12680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15110:x1876702067464960:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 185us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762891.914209:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 298 00000100:00100000:2.0:1789762891.915421:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762891.915444:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067465216 00000100:00100000:3.0:1789762891.915450:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067465216 00000100:00100000:3.0:1789762891.915476:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 299 00000100:00100000:3.0:1789762891.915486:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a13800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15110:x1876702067465216:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:3.0:1789762891.915508:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762891.915520:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932400/0xdf5a7ac59f176901 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c935 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915557:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176908 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915566:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176908 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915581:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176908 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915589:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f17690f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915595:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f17690f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915600:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f17690f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915715:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f176916 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915731:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f176916 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915740:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f176916 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915752:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176908 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915759:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176908 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915767:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176908 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915775:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176908 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915783:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762891.915786:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176908 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915792:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176908 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915799:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762891.915802:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762891.915804:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176908 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915808:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176908 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915823:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f17690f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915827:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f17690f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915842:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f17690f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915845:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762891.915847:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f17690f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915851:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f17690f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915855:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba200) 00010000:00010000:3.0:1789762891.915857:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762891.915859:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f17690f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915862:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f17690f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915880:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5932400/0xdf5a7ac59f176901 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c935 expref: 9 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915888:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f176916 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c935 expref: 8 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.915902:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37e37a7200) 00010000:00010000:3.0:1789762891.915904:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762891.915906:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5933800, rc 0) 00000100:00100000:3.0:1789762891.916417:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a13800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15110:x1876702067465216:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 933us (995us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762891.916422:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 299 00000100:00100000:3.0:1789762891.917435:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762891.917457:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762891.917489:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067465472 00000100:00100000:0.0:1789762891.917494:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067465472 00000100:00100000:2.0:1789762891.917505:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067465600 00000100:00100000:2.0:1789762891.917511:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067465600 00000100:00100000:0.0:1789762891.917528:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 300 00000100:00100000:0.0:1789762891.917540:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15110:x1876702067465472:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762891.917552:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1789762891.917556:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 57 00000100:00100000:2.0:1789762891.917565:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067465600:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762891.917580:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314600/0xdf5a7ac59f17691d lrc: 2/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c93c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.917581:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762891.917595:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5933800/0xdf5a7ac59f176916 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c935 expref: 8 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.917606:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37e37a7200) 00010000:00010000:2.0:1789762891.917608:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00200000:0.0:1789762891.917623:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14:0x0]: returning size 16384 00000100:00100000:2.0:1789762891.917640:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067465600:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 76us (205us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762891.917646:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 57 00010000:00010000:0.0:1789762891.917701:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314600/0xdf5a7ac59f17691d lrc: 1/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c93c expref: 9 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.917710:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:0.0:1789762891.917713:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762891.917716:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314600/0xdf5a7ac59f17691d lrc: 0/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c93c expref: 9 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.917723:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca314600, rc 0) 00000100:00100000:0.0:1789762891.917752:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15110:x1876702067465472:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 217us (294us total) trans 4294967424 rc 301/301 00000100:00100000:0.0:1789762891.917758:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 300 00000100:00100000:3.0:1789762891.919156:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762891.919202:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067465856 00000100:00100000:2.0:1789762891.919206:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067465856 00000100:00100000:2.0:1789762891.919233:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 301 00000100:00100000:2.0:1789762891.919241:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15110:x1876702067465856:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:2.0:1789762891.919286:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17692b lrc: 3/0,1 mode: --/PW res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.919300:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17692b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.919308:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17692b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762891.919446:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000303a63e3 reply state 000000009ee04f00 transno 4294967425 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17692b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.919456:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17692b lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.919462:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:2.0:1789762891.919464:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:2.0:1789762891.919467:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762891.919471:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14:0x0]: returning size 16384 00000100:00100000:2.0:1789762891.919521:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:15110:x1876702067465856:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 281us (365us total) trans 4294967425 rc 0/0 00000100:00100000:2.0:1789762891.919527:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 301 00000100:00100000:2.0:1789762891.920394:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762891.920443:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067466112 00000100:00100000:0.0:1789762891.920450:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067466112 00000100:00100000:0.0:1789762891.920502:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 84 00000100:00100000:0.0:1789762891.920514:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:15110:x1876702067466112:12345-192.168.203.37@tcp:35:lfs.0 00000100:00100000:0.0:1789762891.920676:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15110:x1876702067466112:12345-192.168.203.37@tcp:35:lfs.0 Request processed in 165us (282us total) trans 4294967426 rc 0/0 00000100:00100000:0.0:1789762891.920684:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 84 00000100:00100000:0.0:1789762891.925053:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762891.925086:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067466240 00000100:00100000:3.0:1789762891.925091:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067466240 00000100:00100000:3.0:1789762891.925124:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 302 00000100:00100000:3.0:1789762891.925132:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15111:x1876702067466240:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:3.0:1789762891.925152:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762891.925176:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5932400/0xdf5a7ac59f176932 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c943 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.925217:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f176939 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.925225:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f176939 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.925232:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f176939 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762891.925250:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14:0x0]: returning size 16384 00000004:00200000:3.0:1789762891.925257:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762891.925270:0:6419:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f176939 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.925294:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5932400/0xdf5a7ac59f176932 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c943 expref: 13 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.925307:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f176939 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c943 expref: 12 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.925313:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762891.925315:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789762891.925317:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5932a00, rc 0) 00000100:00100000:3.0:1789762891.925344:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:15111:x1876702067466240:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 214us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762891.925349:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 302 00000100:00100000:2.0:1789762891.926467:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762891.926513:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067466496 00000100:00100000:3.0:1789762891.926519:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067466496 00000100:00100000:3.0:1789762891.926549:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 303 00000100:00100000:3.0:1789762891.926556:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a10a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:15111:x1876702067466496:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:3.0:1789762891.926583:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762891.926593:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5932a00/0xdf5a7ac59f176939 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c943 expref: 12 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.926601:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762891.926604:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762891.926612:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176940 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.926629:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176940 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.926637:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176940 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762891.926733:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e7a5c944 reply state 000000008e2ee564 transno 4294967427 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176940 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.926745:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176940 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.926753:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762891.926755:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:3.0:1789762891.926758:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762891.926763:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14:0x0]: returning size 16384 00000100:00100000:3.0:1789762891.926804:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a10a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:15111:x1876702067466496:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 248us (337us total) trans 4294967427 rc 0/0 00000100:00100000:3.0:1789762891.926809:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 303 00000100:00100000:2.0:1789762891.943884:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762891.943940:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067466624 00000100:00100000:0.0:1789762891.943947:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067466624 00000100:00100000:0.0:1789762891.943983:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 304 00000100:00100000:0.0:1789762891.943991:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a10700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15115:x1876702067466624:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:0.0:1789762891.944005:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762891.944017:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176947 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c94a expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944058:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f17694e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944068:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f17694e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944075:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f17694e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944084:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176955 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944089:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176955 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944093:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176955 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944283:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f17695c lrc: 3/1,0 mode: --/PR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944293:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f17695c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944313:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f17695c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762891.944331:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f17695c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944342:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f17694e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944347:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f17694e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944351:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f17694e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944356:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f17694e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944360:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762891.944363:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f17694e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944368:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f17694e lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944375:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762891.944382:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762891.944386:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f17694e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944390:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f17694e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944398:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176955 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944402:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176955 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944406:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176955 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944410:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762891.944411:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176955 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944415:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176955 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944420:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37e37a7f00) 00010000:00010000:0.0:1789762891.944421:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762891.944423:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176955 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944426:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176955 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944444:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176947 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c94a expref: 9 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944462:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f17695c lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c94a expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762891.944467:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37e37a7200) 00010000:00010000:0.0:1789762891.944469:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762891.944470:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca314e00, rc 0) 00000100:00100000:0.0:1789762891.944508:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a10700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15115:x1876702067466624:12345-192.168.203.37@tcp:101:dd.500 Request processed in 518us (632us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762891.944512:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 304 00000100:00100000:0.0:1789762891.945460:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762891.945510:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067466752 00000100:00100000:2.0:1789762891.945516:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067466752 00000100:00100000:2.0:1789762891.945545:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 305 00000100:00100000:2.0:1789762891.945551:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:15115:x1876702067466752:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:2.0:1789762891.945562:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762891.945571:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f176963 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c951 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.945614:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17696a lrc: 3/1,0 mode: --/PR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.945623:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17696a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.945631:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17696a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762891.945652:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14:0x0]: returning size 16384 00000004:00200000:2.0:1789762891.945657:0:6420:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000100:00100000:0.0:1789762891.945662:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:2.0:1789762891.945686:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768000/0xdf5a7ac59f176963 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c951 expref: 15 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762891.945697:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067466880 00000100:00100000:3.0:1789762891.945704:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067466880 00010000:00010000:2.0:1789762891.945705:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17696a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c951 expref: 14 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.945713:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:2.0:1789762891.945716:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1789762891.945718:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769600, rc 0) 00000100:00100000:2.0:1789762891.945750:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15115:x1876702067466752:12345-192.168.203.37@tcp:101:dd.500 Request processed in 199us (290us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762891.945756:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 305 00000100:00100000:3.0:1789762891.945798:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 58 00000100:00100000:3.0:1789762891.945811:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067466880:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762891.945823:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762891.945837:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f17695c lrc: 0/0,0 mode: --/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c94a expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.945894:0:8070:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37e37a7200) 00010000:00010000:3.0:1789762891.945907:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762891.945965:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067466880:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 157us (301us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762891.945972:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 58 00000100:00100000:0.0:1789762891.946760:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762891.946795:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067467136 00000100:00100000:2.0:1789762891.946799:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067467136 00000100:00100000:2.0:1789762891.946824:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 306 00000100:00100000:2.0:1789762891.946830:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15115:x1876702067467136:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:2.0:1789762891.946848:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762891.946858:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f176971 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c958 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.946944:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176978 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.946956:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176978 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.946965:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176978 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.946978:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17697f lrc: 3/0,1 mode: --/PW res: [0x240000402:0x14:0x0].0x73550ecb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.946983:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17697f lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.946988:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17697f lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762891.947091:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d2a10e00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074715136:0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1789762891.947123:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762891.947145:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252f3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762891.947152:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074715136 00000100:00100000:3.0:1789762891.947167:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074715136 00000100:00100000:3.0:1789762891.947194:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 363 00000100:00100000:3.0:1789762891.947199:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a13480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074715136:12345-0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1789762891.947252:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a13480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:13389:x1876702074715136:12345-0@lo:1000:mdt00_005.0 Request processed in 53us (128us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762891.947258:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 363 00000100:00100000:2.0:1789762891.947275:0:13389:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967425 gen 2 00000004:00080000:2.0:1789762891.947280:0:13389:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967425 committed transno 4294967425 00000100:00100000:2.0:1789762891.947299:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d2a10e00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:13389:1876702074715136:0@lo:1000:mdt00_005.0 00000004:00080000:2.0:1789762891.947557:0:13389:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0xc:0x0], index 0: 0 00000004:00080000:2.0:1789762891.947564:0:13389:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0xc:0x0], index 1: 0 00010000:00010000:2.0:1789762891.947676:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f176986 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947683:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f176986 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947689:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f176986 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947709:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176978 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947713:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176978 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947717:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176978 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947721:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176978 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947725:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762891.947727:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176978 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947731:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176978 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947736:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:2.0:1789762891.947738:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762891.947740:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176978 lrc: 3/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947753:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768200/0xdf5a7ac59f176978 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762891.947760:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003d55d5cc reply state 00000000148f1b57 transno 4294967428 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17697f lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947768:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17697f lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947772:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x73550ecb (ffff9c37d3bee300) 00010000:00010000:2.0:1789762891.947774:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762891.947776:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762891.947792:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769000/0xdf5a7ac59f176971 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c958 expref: 17 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947800:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f176986 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c958 expref: 16 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762891.947805:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:2.0:1789762891.947806:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762891.947808:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769e00, rc 0) 00000100:00100000:2.0:1789762891.947835:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:15115:x1876702067467136:12345-192.168.203.37@tcp:101:dd.500 Request processed in 1005us (1078us total) trans 4294967428 rc 0/0 00000100:00100000:2.0:1789762891.947841:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 306 00000100:00100000:0.0:1789762891.950701:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762891.950735:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067467264 00000100:00100000:3.0:1789762891.950742:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067467264 00000100:00080000:0.0:1789762891.950790:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762891.950797:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 176 00000100:00100000:3.0:1789762891.950806:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:15115:x1876702067467264:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762891.950814:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762891.950825:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f176994 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c95f expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762891.950842:0:9037:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xc:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762891.950854:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=365s v=(0 0) 00010000:00010000:3.0:1789762891.950858:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762891.950865:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f176994 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c95f expref: 6 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.950876:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f176994 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c95f expref: 6 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762891.950910:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5932a00, rc 0) 00000100:00100000:3.0:1789762891.950979:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15115:x1876702067467264:12345-192.168.203.37@tcp:101:dd.500 Request processed in 174us (278us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762891.950985:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 176 00000100:00100000:0.0:1789762891.959492:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762891.959548:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067467392 00000100:00100000:1.0:1789762891.959554:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067467392 00002000:00100000:1.0:1789762891.959595:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xc:0x0] (0->1048575) req@ffff9c37ff69ad80 x1876702067467392/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:672/0 lens 488/0 e 0 to 0 dl 1789762902 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762891.959603:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff69ad80 with x1876702067467392 ext(0->1048575) 00002000:00010000:1.0:1789762891.959607:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff69ad80 00000100:00100000:1.0:1789762891.959620:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340557233944 time 340557334714 state 0x0 ref 1 00000100:00100000:1.0:1789762891.959627:0:8059:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 8: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 340557233944 time 340557334714 state 0x0 ref 1 00000100:00100000:1.0:1789762891.959639:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 3 deadline 365306715956 time 365206715956 state 0x1 ref 1 00000100:00100000:1.0:1789762891.959660:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 365306715956 time 365206738816 state 0x0 ref 1 00000100:00100000:1.0:1789762891.959668:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067467392:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762891.965439:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067467392:12345-192.168.203.37@tcp:4:dd.500 Request processed in 5768us (5947us total) trans 4294967913 rc 0/0 00000100:00100000:1.0:1789762891.965447:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 693 00000100:00100000:1.0:1789762891.965454:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 365306715956 time 365206738816 state 0x2 ref 0 00002000:00100000:1.0:1789762891.965463:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xc:0x0] (0->1048575) req@ffff9c37ff69ad80 x1876702067467392/t4294967913(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:672/0 lens 488/448 e 0 to 0 dl 1789762902 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762891.965471:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff69ad80 with x1876702067467392 ext(0->1048575) 00002000:00010000:1.0:1789762891.965475:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff69ad80 00000100:00100000:0.0:1789762891.967793:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762891.967849:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067467520 00000100:00100000:1.0:1789762891.967855:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067467520 00002000:00100000:1.0:1789762891.967875:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0xc:0x0] (1048576->2097151) req@ffff9c37ff69b800 x1876702067467520/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:672/0 lens 488/0 e 0 to 0 dl 1789762902 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762891.967884:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff69b800 with x1876702067467520 ext(1048576->2097151) 00002000:00010000:1.0:1789762891.967896:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff69b800 00000100:00100000:1.0:1789762891.967904:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 365306715956 time 365206738816 state 0x0 ref 1 00000100:00100000:1.0:1789762891.967912:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 365306738816 time 365206738816 state 0x1 ref 1 00000100:00100000:1.0:1789762891.967987:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 365306738816 time 365215064692 state 0x0 ref 1 00000100:00100000:1.0:1789762891.967997:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067467520:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762891.970601:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2397:x1876702067467520:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2604us (2808us total) trans 4294967914 rc 0/0 00000100:00100000:1.0:1789762891.970608:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 694 00000100:00100000:1.0:1789762891.970613:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 365306738816 time 365215064692 state 0x2 ref 0 00002000:00100000:1.0:1789762891.970631:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0xc:0x0] (1048576->2097151) req@ffff9c37ff69b800 x1876702067467520/t4294967914(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:672/0 lens 488/448 e 0 to 0 dl 1789762902 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762891.970638:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff69b800 with x1876702067467520 ext(1048576->2097151) 00002000:00010000:1.0:1789762891.970642:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff69b800 00000100:00100000:0.0:1789762891.976823:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762891.976864:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067467648 00000100:00100000:3.0:1789762891.976871:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067467648 00002000:00100000:3.0:1789762891.976897:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0xc:0x0] (2097152->3145727) req@ffff9c37ff698700 x1876702067467648/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:672/0 lens 488/0 e 0 to 0 dl 1789762902 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762891.976908:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff698700 with x1876702067467648 ext(2097152->3145727) 00002000:00010000:3.0:1789762891.976913:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff698700 00000100:00100000:3.0:1789762891.976946:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 365306738816 time 365215064692 state 0x0 ref 1 00000100:00100000:3.0:1789762891.976957:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 365315064692 time 365215064692 state 0x1 ref 1 00000100:00100000:3.0:1789762891.976979:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365315064692 time 365224057594 state 0x0 ref 1 00000100:00100000:3.0:1789762891.976990:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2398:x1876702067467648:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762891.979093:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:2398:x1876702067467648:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2105us (2270us total) trans 4294967915 rc 0/0 00000100:00100000:3.0:1789762891.979099:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 695 00000100:00100000:3.0:1789762891.979104:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365315064692 time 365224057594 state 0x2 ref 0 00002000:00100000:3.0:1789762891.979120:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0xc:0x0] (2097152->3145727) req@ffff9c37ff698700 x1876702067467648/t4294967915(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:672/0 lens 488/448 e 0 to 0 dl 1789762902 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762891.979126:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff698700 with x1876702067467648 ext(2097152->3145727) 00002000:00010000:3.0:1789762891.979129:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff698700 00000100:00100000:0.0:1789762891.985466:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762891.985507:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762891.985525:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067467776 00000100:00100000:1.0:1789762891.985533:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067467776 00000100:00100000:3.0:1789762891.985541:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067467904 00000100:00100000:3.0:1789762891.985548:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067467904 00002000:00100000:1.0:1789762891.985561:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xc:0x0] (3145728->4194303) req@ffff9c37ff699180 x1876702067467776/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:672/0 lens 488/0 e 0 to 0 dl 1789762902 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762891.985582:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff699180 with x1876702067467776 ext(3145728->4194303) 00002000:00010000:1.0:1789762891.985586:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff699180 00000100:00100000:1.0:1789762891.985598:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365315064692 time 365224057594 state 0x0 ref 1 00000100:00100000:3.0:1789762891.985609:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 177 00000100:00100000:1.0:1789762891.985616:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365324057594 time 365224057594 state 0x1 ref 1 00000100:00100000:3.0:1789762891.985621:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0+-99:2394:x1876702067467904:12345-192.168.203.37@tcp:8:dd.500 00010000:00080000:3.0:1789762891.985647:0:9038:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp t0 exp 0000000000000000 cur 365 last 0 00000020:00000080:3.0:1789762891.985673:0:9038:0:(genops.c:1385:class_connect()) connect: client fa6fc127-0d58-47ca-82d1-2c8ad7b086c1, cookie 0xdf5a7ac59f17699b 00002000:00100000:3.0:1789762891.985682:0:9038:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli fa6fc127-0d58-47ca-82d1-2c8ad7b086c1/ffff9c37e1bee000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:3.0:1789762891.985697:0:9038:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:3.0:1789762891.985820:0:9038:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:3.0:1789762891.985842:0:9038:0:(import.c:65:import_set_state_nolock()) ffff9c37e1bef800 : changing import state from RECOVER to FULL 00000100:00100000:3.0:1789762891.985876:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2394:x1876702067467904:12345-192.168.203.37@tcp:8:dd.500 Request processed in 258us (368us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762891.985883:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 177 00000100:00100000:0.0:1789762891.986842:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762891.986899:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067468032 00000100:00100000:1.0:1789762891.986905:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067468032 00000100:00080000:0.0:1789762891.986960:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762891.986975:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 178 00000100:00100000:1.0:1789762891.986984:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15115:x1876702067468032:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:1.0:1789762891.986993:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762891.987008:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000496311b/0xdf5a7ac59f1769a2 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c966 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762891.987040:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0xc:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789762891.987053:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xc:0x0].0x0 since=365s v=(0 0) 00010000:00010000:1.0:1789762891.987057:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762891.987064:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000496311b/0xdf5a7ac59f1769a2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c966 expref: 7 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762891.987077:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000496311b/0xdf5a7ac59f1769a2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c966 expref: 7 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762891.987086:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c762ca00, rc 0) 00000100:00100000:1.0:1789762891.987124:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15115:x1876702067468032:12345-192.168.203.37@tcp:101:dd.500 Request processed in 143us (284us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762891.987129:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 178 00000100:00100000:2.0:1789762891.994938:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762891.994984:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067468160 00000100:00100000:1.0:1789762891.994990:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067468160 00002000:00100000:1.0:1789762891.995009:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0xc:0x0] (0->1048575) req@ffff9c36d2a10e00 x1876702067468160/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:672/0 lens 488/0 e 0 to 0 dl 1789762902 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762891.995017:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d2a10e00 with x1876702067468160 ext(0->1048575) 00002000:00010000:1.0:1789762891.995020:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36d2a10e00 00000100:00100000:1.0:1789762891.995029:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365324057594 time 365224057594 state 0x1 ref 2 00000100:00100000:0.0:1789762892.002616:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762892.002661:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067468288 00000100:00100000:1.0:1789762892.002667:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067468288 00002000:00100000:1.0:1789762892.002686:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0xc:0x0] (1048576->2097151) req@ffff9c37ff699500 x1876702067468288/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762892.002694:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff699500 with x1876702067468288 ext(1048576->2097151) 00002000:00010000:1.0:1789762892.002697:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ff699500 00000100:00100000:1.0:1789762892.002706:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365324057594 time 365224057594 state 0x1 ref 3 00000100:00100000:3.0:1789762892.077116:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365424181246 time 365324181246 state 0x1 ref 3 00000100:00100000:3.0:1789762892.077142:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:2397:x1876702067467776:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762892.080597:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:2397:x1876702067467776:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3463us (95130us total) trans 4294967916 rc 0/0 00000100:00100000:3.0:1789762892.080608:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 696 00002000:00100000:3.0:1789762892.080637:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0xc:0x0] (3145728->4194303) req@ffff9c37ff699180 x1876702067467776/t4294967916(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:672/0 lens 488/448 e 0 to 0 dl 1789762902 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762892.080645:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff699180 with x1876702067467776 ext(3145728->4194303) 00002000:00010000:3.0:1789762892.080650:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff699180 00000100:00100000:1.0:1789762892.177250:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365524315834 time 365424315834 state 0x1 ref 2 00000100:00100000:1.0:1789762892.177284:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a10e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067468160:12345-192.168.203.37@tcp:4:dd.500 00000001:00100000:0.0:1789762892.190327:0:9000:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000001:00080000:3.0:1789762892.201263:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967400 is committed 00000100:00100000:0.0:1789762892.201332:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a10e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067468160:12345-192.168.203.37@tcp:4:dd.500 Request processed in 24063us (206390us total) trans 4294967400 rc 0/0 00000100:00100000:0.0:1789762892.201341:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 697 00002000:00100000:0.0:1789762892.201360:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0xc:0x0] (0->1048575) req@ffff9c36d2a10e00 x1876702067468160/t4294967400(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:672/0 lens 488/448 e 0 to 0 dl 1789762902 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762892.201372:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d2a10e00 with x1876702067468160 ext(0->1048575) 00002000:00010000:0.0:1789762892.201380:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36d2a10e00 00000100:00100000:0.0:1789762892.203831:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.203878:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067468416 00000100:00100000:3.0:1789762892.203886:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067468416 00002000:00100000:3.0:1789762892.203937:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0xc:0x0] (2097152->3145727) req@ffff9c37ff698000 x1876702067468416/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762892.203950:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff698000 with x1876702067468416 ext(2097152->3145727) 00002000:00010000:3.0:1789762892.203955:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ff698000 00000100:00100000:3.0:1789762892.203975:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365524315834 time 365424315834 state 0x1 ref 2 00000100:00100000:3.0:1789762892.210693:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762892.210786:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067468544 00000100:00100000:2.0:1789762892.210791:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067468544 00002000:00100000:2.0:1789762892.210810:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0xc:0x0] (3145728->4194303) req@ffff9c37d1ae0000 x1876702067468544/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762892.210825:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1ae0000 with x1876702067468544 ext(3145728->4194303) 00002000:00010000:2.0:1789762892.210828:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d1ae0000 00000100:00100000:2.0:1789762892.210846:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365524315834 time 365424315834 state 0x1 ref 3 00000100:00100000:2.0:1789762892.217196:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762892.217246:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067468672 00000100:00100000:0.0:1789762892.217252:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067468672 00002000:00100000:0.0:1789762892.217273:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0xc:0x0] (4194304->5242879) req@ffff9c37d1aba300 x1876702067468672/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762892.217281:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1aba300 with x1876702067468672 ext(4194304->5242879) 00002000:00010000:0.0:1789762892.217284:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d1aba300 00000100:00100000:0.0:1789762892.217293:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365524315834 time 365424315834 state 0x1 ref 4 00000100:00100000:0.0:1789762892.223842:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762892.223889:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067468800 00000100:00100000:1.0:1789762892.223895:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067468800 00002000:00100000:1.0:1789762892.223915:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xc:0x0] (5242880->6291455) req@ffff9c37ff69b480 x1876702067468800/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762892.223938:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff69b480 with x1876702067468800 ext(5242880->6291455) 00002000:00010000:1.0:1789762892.223941:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff69b480 00000100:00100000:1.0:1789762892.223954:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365524315834 time 365424315834 state 0x1 ref 5 00000100:00100000:3.0:1789762892.230422:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762892.230463:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067468928 00000100:00100000:1.0:1789762892.230467:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067468928 00002000:00100000:1.0:1789762892.230484:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xc:0x0] (6291456->7340031) req@ffff9c37d1ae3480 x1876702067468928/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762892.230491:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1ae3480 with x1876702067468928 ext(6291456->7340031) 00002000:00010000:1.0:1789762892.230494:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d1ae3480 00000100:00100000:1.0:1789762892.230501:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365524315834 time 365424315834 state 0x1 ref 6 00000100:00100000:2.0:1789762892.237167:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762892.237220:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067469056 00000100:00100000:1.0:1789762892.237224:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067469056 00002000:00100000:1.0:1789762892.237241:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0xc:0x0] (7340032->8388607) req@ffff9c37d1abad80 x1876702067469056/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762892.237252:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1abad80 with x1876702067469056 ext(7340032->8388607) 00002000:00010000:1.0:1789762892.237255:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d1abad80 00000100:00100000:1.0:1789762892.237263:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365524315834 time 365424315834 state 0x1 ref 7 00000100:00100000:3.0:1789762892.277312:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365624379566 time 365524379566 state 0x1 ref 7 00000100:00100000:3.0:1789762892.277334:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067468288:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762892.280287:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2397:x1876702067468288:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2958us (277668us total) trans 4294967401 rc 0/0 00000100:00100000:0.0:1789762892.280298:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 698 00002000:00100000:0.0:1789762892.280309:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0xc:0x0] (1048576->2097151) req@ffff9c37ff699500 x1876702067468288/t4294967401(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762892.280320:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff699500 with x1876702067468288 ext(1048576->2097151) 00002000:00010000:0.0:1789762892.280327:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ff699500 00000100:00100000:2.0:1789762892.282242:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762892.282335:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067469184 00000100:00100000:1.0:1789762892.282343:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067469184 00002000:00100000:1.0:1789762892.282370:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0xc:0x0] (4194304->5242879) req@ffff9c37fd9f6d80 x1876702067469184/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762892.282378:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fd9f6d80 with x1876702067469184 ext(4194304->5242879) 00002000:00010000:1.0:1789762892.282381:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fd9f6d80 00000100:00100000:1.0:1789762892.282393:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365624379566 time 365524379566 state 0x1 ref 7 00000100:00100000:2.0:1789762892.288064:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074715264:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762892.288102:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762892.288153:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074715264 00000100:00100000:0.0:1789762892.288170:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074715264 00000100:00100000:0.0:1789762892.288199:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:0.0:1789762892.288208:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074715264:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762892.288246:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762892.288288:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074715264:12345-0@lo:41:osp-pre-1-0.0 Request processed in 80us (184us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762892.288296:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:2.0:1789762892.288332:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074715264:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762892.289197:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762892.289276:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067469312 00000100:00100000:1.0:1789762892.289281:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067469312 00002000:00100000:1.0:1789762892.289300:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0xc:0x0] (5242880->6291455) req@ffff9c37d1ae1c00 x1876702067469312/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762892.289308:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1ae1c00 with x1876702067469312 ext(5242880->6291455) 00002000:00010000:1.0:1789762892.289311:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d1ae1c00 00000100:00100000:1.0:1789762892.289320:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365624379566 time 365524379566 state 0x1 ref 8 00000100:00100000:1.0:1789762892.377403:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365724468683 time 365624468683 state 0x1 ref 8 00000100:00100000:1.0:1789762892.377429:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:2398:x1876702067468416:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762892.380687:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:2398:x1876702067468416:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3268us (176855us total) trans 4294967402 rc 0/0 00000100:00100000:1.0:1789762892.380695:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 699 00002000:00100000:1.0:1789762892.380705:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0xc:0x0] (2097152->3145727) req@ffff9c37ff698000 x1876702067468416/t4294967402(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762892.380713:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff698000 with x1876702067468416 ext(2097152->3145727) 00002000:00010000:1.0:1789762892.380719:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37ff698000 00000100:00100000:3.0:1789762892.382997:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762892.383078:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067469440 00000100:00100000:0.0:1789762892.383087:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067469440 00002000:00100000:0.0:1789762892.383116:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0xc:0x0] (6291456->7340031) req@ffff9c37fdfe1c00 x1876702067469440/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762892.383123:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfe1c00 with x1876702067469440 ext(6291456->7340031) 00002000:00010000:0.0:1789762892.383126:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fdfe1c00 00000100:00100000:0.0:1789762892.383141:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365724468683 time 365624468683 state 0x1 ref 8 00000100:00100000:2.0:1789762892.389697:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762892.389722:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.389781:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067469824 00000100:00100000:3.0:1789762892.389788:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067469824 00000100:00100000:1.0:1789762892.389808:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067469568 00000100:00100000:1.0:1789762892.389814:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067469568 00000100:00100000:3.0:1789762892.389829:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 85 00002000:00100000:1.0:1789762892.389838:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0xc:0x0] (7340032->8388607) req@ffff9c37fd9f4e00 x1876702067469568/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00000100:00100000:3.0:1789762892.389842:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:15115:x1876702067469824:12345-192.168.203.37@tcp:35:dd.500 00002000:00100000:1.0:1789762892.389856:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fd9f4e00 with x1876702067469568 ext(7340032->8388607) 00002000:00010000:1.0:1789762892.389860:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fd9f4e00 00000100:00100000:1.0:1789762892.389870:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365724468683 time 365624468683 state 0x1 ref 9 00000001:00080000:3.0:1789762892.390064:0:13186:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967429 00000100:00100000:3.0:1789762892.390141:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:15115:x1876702067469824:12345-192.168.203.37@tcp:35:dd.500 Request processed in 304us (445us total) trans 4294967430 rc 0/0 00000100:00100000:3.0:1789762892.390147:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 85 00000020:00080000:3.0:1789762892.390150:0:13186:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1789762892.416011:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762892.418425:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762892.418502:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067470080 00000100:00100000:2.0:1789762892.418511:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067470080 00000100:00100000:2.0:1789762892.418559:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 307 00000100:00100000:2.0:1789762892.418571:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:15126:x1876702067470080:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762892.418593:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762892.418613:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f1769a9 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c96d expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.418712:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f1769b0 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.418734:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f1769b0 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.418760:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f1769b0 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.418772:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f1769b7 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.418777:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f1769b7 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.418782:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f1769b7 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762892.419073:0:11692:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0xd:0x0], index 1: 0 00000004:00080000:2.0:1789762892.419078:0:11692:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0xd:0x0], index 0: 0 00010000:00010000:2.0:1789762892.419212:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769400/0xdf5a7ac59f1769be lrc: 3/1,0 mode: --/PR res: [0x240000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419218:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769400/0xdf5a7ac59f1769be lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419224:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769400/0xdf5a7ac59f1769be lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419248:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f1769b0 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419253:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f1769b0 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419258:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f1769b0 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419271:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f1769b0 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419275:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762892.419278:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f1769b0 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419283:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f1769b0 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419293:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:2.0:1789762892.419295:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762892.419298:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f1769b0 lrc: 3/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419303:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768e00/0xdf5a7ac59f1769b0 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762892.419312:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007818de60 reply state 00000000a69b0851 transno 4294967431 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f1769b7 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419319:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f1769b7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419323:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x2b0e463d (ffff9c37d3bee200) 00010000:00010000:2.0:1789762892.419325:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762892.419326:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762892.419345:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768a00/0xdf5a7ac59f1769a9 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c96d expref: 22 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419365:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769400/0xdf5a7ac59f1769be lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c96d expref: 21 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.419371:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16:0x0].0x0 (ffff9c37d3bee900) 00010000:00010000:2.0:1789762892.419372:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762892.419374:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769400, rc 0) 00000100:00100000:2.0:1789762892.419421:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:15126:x1876702067470080:12345-192.168.203.37@tcp:101:dd.0 Request processed in 852us (999us total) trans 4294967431 rc 0/0 00000100:00100000:2.0:1789762892.419429:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 307 00000100:00100000:2.0:1789762892.425092:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.425180:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067470208 00000100:00100000:2.0:1789762892.425183:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.425186:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067470208 00000100:00100000:2.0:1789762892.425222:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762892.425250:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067470336 00000100:00100000:0.0:1789762892.425255:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067470336 00000100:00100000:1.0:1789762892.425263:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067470464 00000100:00100000:1.0:1789762892.425269:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067470464 00000100:00100000:2.0:1789762892.425282:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 179 00000100:00100000:3.0:1789762892.425293:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 180 00000100:00100000:2.0:1789762892.425296:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:15126:x1876702067470208:12345-192.168.203.37@tcp:101:dd.0 00000100:00100000:0.0:1789762892.425303:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 181 00000100:00100000:3.0:1789762892.425308:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:15128:x1876702067470336:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762892.425311:0:8052:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1789762892.425315:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:15127:x1876702067470464:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762892.425320:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762892.425325:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762892.425330:0:8052:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c974 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.425331:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f1769d3 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c97b expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.425337:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f1769da lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c982 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762892.425364:0:8052:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0xd:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789762892.425380:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=365s v=(0 0) 00010000:00010000:2.0:1789762892.425386:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762892.425396:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 20 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.425412:0:9037:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 20 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.425420:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762892.425423:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789762892.425426:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f1769d3 lrc: 4/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c97b expref: 20 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.425434:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762892.425439:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(2 0) 00010000:00010000:0.0:1789762892.425445:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f1769da lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c982 expref: 20 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.425460:0:8052:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 20 pid: 8052 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.425478:0:8052:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 20 pid: 8052 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.425480:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f1769da lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c982 expref: 20 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.425491:0:8052:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768a00, rc 0) 00010000:00010000:0.0:1789762892.425494:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca314400, rc 0) 00010000:00010000:3.0:1789762892.425499:0:9037:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 20 pid: 8052 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.425509:0:9037:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### not re-adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 20 pid: 8052 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762892.425533:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:15127:x1876702067470464:12345-192.168.203.37@tcp:101:dd.0 Request processed in 220us (308us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762892.425533:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:15126:x1876702067470208:12345-192.168.203.37@tcp:101:dd.0 Request processed in 239us (440us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762892.425536:0:9037:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fdfe0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9037:1876702074715392:192.168.203.37@tcp:104: 00000100:00100000:2.0:1789762892.425547:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 179 00000100:00100000:0.0:1789762892.425552:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 181 00000100:00100000:3.0:1789762892.425619:0:9037:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d2569600 going to sleep for 11 seconds 00000100:00100000:3.0:1789762892.426953:0:9037:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fdfe0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9037:1876702074715392:192.168.203.37@tcp:104: 00010000:00010000:3.0:1789762892.426986:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f1769d3 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c97b expref: 18 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.426996:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5932400, rc 0) 00000100:00100000:3.0:1789762892.427022:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:15128:x1876702067470336:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1721us (1842us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762892.427028:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 180 00000100:00100000:1.0:1789762892.429980:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.430020:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067470592 00000100:00100000:3.0:1789762892.430034:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067470592 00000100:00100000:3.0:1789762892.430058:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 182 00000100:00100000:3.0:1789762892.430065:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:15132:x1876702067470592:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762892.430072:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762892.430082:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c989 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762892.430097:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xd:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762892.430106:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=365s v=(0 0) 00010000:00010000:3.0:1789762892.430108:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762892.430113:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 19 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.430133:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 19 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.430140:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5933600, rc 0) 00000100:00100000:3.0:1789762892.430182:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:15132:x1876702067470592:12345-192.168.203.37@tcp:101:dd.0 Request processed in 117us (205us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762892.430189:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 182 00000100:00100000:2.0:1789762892.431003:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762892.431059:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067470720 00000100:00100000:0.0:1789762892.431064:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067470720 00000100:00100000:0.0:1789762892.431093:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 183 00000100:00100000:0.0:1789762892.431102:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:15131:x1876702067470720:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762892.431111:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762892.431121:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f1769e8 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c990 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.431141:0:8053:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 20 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.431150:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762892.431165:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762892.431169:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f1769e8 lrc: 4/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c990 expref: 20 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.431201:0:8053:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 20 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.431208:0:8053:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 20 pid: 9038 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762892.431223:0:8053:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff699f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074715520:192.168.203.37@tcp:104: 00000100:00100000:0.0:1789762892.431267:0:8053:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5b1c300 going to sleep for 11 seconds 00000100:00100000:0.0:1789762892.432059:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762892.432066:0:8053:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff699f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074715520:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762892.432085:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f1769e8 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c990 expref: 20 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762892.432095:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067470848 00010000:00010000:2.0:1789762892.432100:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca315000, rc 0) 00000100:00100000:3.0:1789762892.432101:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067470848 00000100:00100000:2.0:1789762892.432137:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:15131:x1876702067470720:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1035us (1133us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762892.432146:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 183 00000100:00100000:2.0:1789762892.432196:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 184 00000100:00100000:2.0:1789762892.432206:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:15129:x1876702067470848:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762892.432215:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762892.432225:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f28035ec/0xdf5a7ac59f1769ef lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c997 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.432242:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(2 0) 00010000:00010000:2.0:1789762892.432261:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762892.432267:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f28035ec/0xdf5a7ac59f1769ef lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c997 expref: 19 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.432278:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f28035ec/0xdf5a7ac59f1769ef lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c997 expref: 19 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.432285:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769200, rc 0) 00000100:00100000:2.0:1789762892.432311:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:15129:x1876702067470848:12345-192.168.203.37@tcp:101:dd.0 Request processed in 108us (255us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762892.432316:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 184 00000100:00100000:0.0:1789762892.437198:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.437249:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067470976 00000100:00100000:3.0:1789762892.437254:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067470976 00000100:00100000:3.0:1789762892.437279:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 185 00000100:00100000:3.0:1789762892.437285:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:15130:x1876702067470976:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762892.437293:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762892.437301:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f1769f6 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c99e expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.437391:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789762892.437395:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(2 0) 00010000:00010000:3.0:1789762892.437401:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f1769f6 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c99e expref: 21 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.437410:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f1769f6 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c99e expref: 21 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.437429:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5933c00, rc 0) 00000100:00100000:0.0:1789762892.437459:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.437463:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:15130:x1876702067470976:12345-192.168.203.37@tcp:101:dd.0 Request processed in 174us (263us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762892.437469:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 185 00000100:00100000:3.0:1789762892.437483:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067471104 00000100:00100000:3.0:1789762892.437486:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067471104 00000100:00100000:2.0:1789762892.437519:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 186 00000100:00100000:2.0:1789762892.437532:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:15136:x1876702067471104:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762892.437543:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762892.437553:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f037a74/0xdf5a7ac59f1769fd lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9a5 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.437574:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762892.437579:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762892.437585:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002f037a74/0xdf5a7ac59f1769fd lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9a5 expref: 20 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.437597:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f037a74/0xdf5a7ac59f1769fd lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9a5 expref: 20 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.437606:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768c00, rc 0) 00000100:00100000:2.0:1789762892.437643:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:15136:x1876702067471104:12345-192.168.203.37@tcp:101:dd.0 Request processed in 114us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762892.437651:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 186 00000100:00100000:1.0:1789762892.441997:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762892.442050:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067471232 00000100:00100000:2.0:1789762892.442055:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067471232 00000100:00100000:2.0:1789762892.442083:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 187 00000100:00100000:2.0:1789762892.442090:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:15135:x1876702067471232:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762892.442098:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762892.442108:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000027c972f9/0xdf5a7ac59f176a04 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9ac expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.442122:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762892.442126:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(5 0) 00010000:00010000:2.0:1789762892.442132:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000027c972f9/0xdf5a7ac59f176a04 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 7 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9ac expref: 21 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.442144:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000027c972f9/0xdf5a7ac59f176a04 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 7 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9ac expref: 21 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.442168:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769c00, rc 0) 00000100:00100000:2.0:1789762892.442204:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:15135:x1876702067471232:12345-192.168.203.37@tcp:101:dd.0 Request processed in 114us (210us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762892.442211:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 187 00000100:00100000:0.0:1789762892.443856:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.443898:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067471360 00000100:00100000:3.0:1789762892.443903:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067471360 00000100:00100000:3.0:1789762892.443974:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 188 00000100:00100000:3.0:1789762892.443981:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:15139:x1876702067471360:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762892.443989:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762892.443997:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f176a0b lrc: 2/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9b3 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.444010:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xd:0x0].0x0 since=0s v=(2 0) 00010000:00010000:3.0:1789762892.444014:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(3 0) 00010000:00010000:3.0:1789762892.444018:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f176a0b lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 5 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9b3 expref: 22 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.444025:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f176a0b lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 5 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9b3 expref: 22 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.444032:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5933800, rc 0) 00000100:00100000:3.0:1789762892.444057:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:15139:x1876702067471360:12345-192.168.203.37@tcp:101:dd.0 Request processed in 78us (202us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762892.444073:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 188 00000100:00100000:0.0:1789762892.444808:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762892.444861:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067471488 00000100:00100000:2.0:1789762892.444865:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067471488 00000100:00100000:2.0:1789762892.444890:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 189 00000100:00100000:2.0:1789762892.444896:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:15140:x1876702067471488:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762892.444910:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762892.444936:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176a12 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9ba expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.444950:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xd:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762892.444953:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762892.444957:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176a12 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9ba expref: 23 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.444965:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176a12 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9ba expref: 23 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.444972:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1769800, rc 0) 00000100:00100000:2.0:1789762892.444999:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:15140:x1876702067471488:12345-192.168.203.37@tcp:101:dd.0 Request processed in 102us (191us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762892.445006:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 189 00000100:00100000:1.0:1789762892.447586:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762892.447654:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762892.447679:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067471616 00000100:00100000:2.0:1789762892.447685:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067471616 00000100:00100000:3.0:1789762892.447695:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067471744 00000100:00100000:3.0:1789762892.447701:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067471744 00000100:00100000:2.0:1789762892.447718:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 190 00000100:00100000:2.0:1789762892.447727:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:15138:x1876702067471616:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762892.447735:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762892.447744:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a19 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9c1 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.447759:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xd:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762892.447764:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(5 0) 00000100:00100000:3.0:1789762892.447768:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 191 00000100:00100000:3.0:1789762892.447777:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:15133:x1876702067471744:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762892.447784:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a19 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 7 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9c1 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.447786:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762892.447796:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb2b668/0xdf5a7ac59f176a20 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9c8 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.447803:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a19 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9c1 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.447813:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xd:0x0].0x0 since=0s v=(5 0) 00010000:00010000:3.0:1789762892.447818:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(6 0) 00010000:00010000:3.0:1789762892.447823:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb2b668/0xdf5a7ac59f176a20 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 8 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9c8 expref: 26 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.447834:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768000, rc 0) 00010000:00010000:3.0:1789762892.447834:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008fb2b668/0xdf5a7ac59f176a20 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 8 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9c8 expref: 26 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762892.447876:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:15138:x1876702067471616:12345-192.168.203.37@tcp:101:dd.0 Request processed in 149us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762892.447884:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 190 00010000:00010000:3.0:1789762892.447955:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36c9af7000, rc 0) 00000100:00100000:3.0:1789762892.447984:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:15133:x1876702067471744:12345-192.168.203.37@tcp:101:dd.0 Request processed in 208us (329us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762892.447991:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 191 00000100:00100000:1.0:1789762892.448080:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762892.448120:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067471872 00000100:00100000:2.0:1789762892.448124:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067471872 00000100:00100000:2.0:1789762892.448148:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 192 00000100:00100000:2.0:1789762892.448166:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:15141:x1876702067471872:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762892.448173:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762892.448190:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176a27 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9cf expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.448202:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xd:0x0].0x0 since=0s v=(6 0) 00010000:00010000:2.0:1789762892.448205:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.448209:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176a27 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9cf expref: 26 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.448216:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176a27 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9cf expref: 26 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.448222:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768200, rc 0) 00000100:00100000:2.0:1789762892.448265:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:15141:x1876702067471872:12345-192.168.203.37@tcp:101:dd.0 Request processed in 111us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762892.448270:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 192 00000100:00100000:0.0:1789762892.449940:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.449981:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067472000 00000100:00100000:3.0:1789762892.449986:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067472000 00000100:00100000:3.0:1789762892.450010:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 193 00000100:00100000:3.0:1789762892.450016:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:15134:x1876702067472000:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762892.450023:0:8052:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762892.450037:0:8052:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005926da61/0xdf5a7ac59f176a2e lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9d6 expref: -99 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.450050:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(5 0) 00010000:00010000:3.0:1789762892.450053:0:8052:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(6 0) 00010000:00010000:3.0:1789762892.450057:0:8052:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005926da61/0xdf5a7ac59f176a2e lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 8 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9d6 expref: 22 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.450064:0:8052:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005926da61/0xdf5a7ac59f176a2e lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 8 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9d6 expref: 22 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.450072:0:8052:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36c9af6000, rc 0) 00000100:00100000:3.0:1789762892.450096:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:15134:x1876702067472000:12345-192.168.203.37@tcp:101:dd.0 Request processed in 81us (158us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762892.450104:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 193 00000100:00100000:1.0:1789762892.451908:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.451948:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067472128 00000100:00100000:3.0:1789762892.451954:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067472128 00000100:00100000:3.0:1789762892.451988:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 194 00000100:00100000:3.0:1789762892.451995:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:15137:x1876702067472128:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762892.452002:0:8054:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762892.452008:0:8054:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000054047fbc/0xdf5a7ac59f176a35 lrc: 2/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9dd expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.452019:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(6 0) 00010000:00010000:3.0:1789762892.452022:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:3.0:1789762892.452026:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000054047fbc/0xdf5a7ac59f176a35 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9dd expref: 23 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.452043:0:8054:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000054047fbc/0xdf5a7ac59f176a35 lrc: 3/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9dd expref: 23 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.452049:0:8054:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36c9af7e00, rc 0) 00000100:00100000:3.0:1789762892.452073:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:15137:x1876702067472128:12345-192.168.203.37@tcp:101:dd.0 Request processed in 81us (168us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762892.452078:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 194 00000100:00100000:1.0:1789762892.453960:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.453994:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067472256 00000100:00100000:3.0:1789762892.453999:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067472256 00002000:00100000:3.0:1789762892.454021:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xd:0x0] (2097152->3145727) req@ffff9c37f6088a80 x1876702067472256/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762892.454028:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f6088a80 with x1876702067472256 ext(2097152->3145727) 00010000:00010000:3.0:1789762892.454033:0:8059:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 26 pid: 9038 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762892.454042:0:8059:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 26 pid: 9038 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762892.454048:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9c37f6088a80 00000100:00100000:3.0:1789762892.454062:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 334752788962 time 334784826762 state 0x0 ref 1 00000100:00100000:3.0:1789762892.454068:0:8059:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 8: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 334752788962 time 334784826762 state 0x0 ref 1 00000100:00100000:3.0:1789762892.454082:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763200 gen 0 rate 20 depth 3 token 3 deadline 365751159135 time 365701159135 state 0x1 ref 1 00000100:00100000:3.0:1789762892.454110:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763200 gen 0 rate 20 depth 3 token 2 deadline 365751159135 time 365701187388 state 0x0 ref 1 00000100:00100000:3.0:1789762892.454117:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2397:x1876702067472256:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762892.458335:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2397:x1876702067472256:12345-192.168.203.37@tcp:4:dd.0 Request processed in 4212us (4370us total) trans 4294967917 rc 0/0 00000100:00100000:0.0:1789762892.458346:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 6 00000100:00100000:0.0:1789762892.458356:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763200 gen 0 rate 20 depth 3 token 2 deadline 365751159135 time 365701187388 state 0x2 ref 0 00002000:00100000:0.0:1789762892.458369:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xd:0x0] (2097152->3145727) req@ffff9c37f6088a80 x1876702067472256/t4294967917(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762892.458379:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f6088a80 with x1876702067472256 ext(2097152->3145727) 00010000:00010000:0.0:1789762892.458387:0:8059:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 28 pid: 9038 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.458400:0:8059:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 28 pid: 9038 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762892.458408:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9c37f6088a80 00000100:00100000:1.0:1789762892.459659:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762892.459737:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067472384 00000100:00100000:2.0:1789762892.459742:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067472384 00010000:00010000:2.0:1789762892.459763:0:6404:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 28 pid: 9038 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762892.459790:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 83 00000100:00100000:2.0:1789762892.459796:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2396:x1876702067472384:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762892.459805:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1789762892.459815:0:6404:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xd:0x0] updating lvb size from disk: 0 -> 3145728 00002000:00010000:2.0:1789762892.459817:0:6404:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xd:0x0] updating lvb mtime from disk: 0 -> 1789762892 00002000:00010000:2.0:1789762892.459819:0:6404:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xd:0x0] updating lvb ctime from disk: 0 -> 1789762892 00010000:00010000:2.0:1789762892.459823:0:6404:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 28 pid: 9038 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459832:0:6404:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 28 pid: 9038 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459848:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000228e839a/0xdf5a7ac59f1769e1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c989 expref: 28 pid: 9038 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459859:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.459862:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.459868:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f1769e8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c990 expref: 27 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459881:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f1769e8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c990 expref: 27 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459888:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.459890:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.459894:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f1769f6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c99e expref: 27 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459899:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f1769f6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c99e expref: 27 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459903:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.459905:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.459908:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f176a0b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9b3 expref: 27 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459913:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f176a0b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9b3 expref: 27 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459946:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.459948:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.459953:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176a12 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9ba expref: 27 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459960:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176a12 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9ba expref: 27 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459971:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.459973:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.459977:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a19 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9c1 expref: 27 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459983:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a19 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9c1 expref: 27 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459987:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.459989:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.459992:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008fb2b668/0xdf5a7ac59f176a20 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9c8 expref: 27 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.459997:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb2b668/0xdf5a7ac59f176a20 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9c8 expref: 27 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.460001:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.460002:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762892.460005:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176a27 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9cf expref: 27 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.460010:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176a27 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9cf expref: 27 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.460035:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176a27 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9cf expref: 27 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762892.460050:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff699f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074715648:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762892.460108:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000008fb2b668/0xdf5a7ac59f176a20 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9c8 expref: 27 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762892.460116:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074715776:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762892.460142:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a19 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9c1 expref: 27 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762892.460148:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074715904:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762892.460189:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176a12 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9ba expref: 27 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762892.460198:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074716032:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762892.460221:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f176a0b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9b3 expref: 27 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762892.460228:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074716160:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762892.460244:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f1769f6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c99e expref: 27 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762892.460250:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074716288:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762892.460281:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f1769e8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c990 expref: 27 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762892.460301:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf5f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074716416:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762892.460318:0:6404:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252f900 going to sleep for 11 seconds 00000100:00100000:1.0:1789762892.461294:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff699f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074715648:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762892.461302:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074715776:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762892.461307:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074715904:192.168.203.37@tcp:105: 00000100:00100000:1.0:1789762892.461313:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074716160:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762892.461481:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074716032:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762892.461490:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074716288:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762892.461576:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf5f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6404:1876702074716416:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762892.461598:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762892.461627:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2396:x1876702067472384:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 1829us (1968us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762892.461635:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 83 00000100:00100000:1.0:1789762892.477467:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365824525950 time 365724525950 state 0x1 ref 9 00000100:00100000:1.0:1789762892.477491:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:2398:x1876702067468544:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762892.484065:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:2398:x1876702067468544:12345-192.168.203.37@tcp:4:dd.500 Request processed in 6580us (273370us total) trans 4294967403 rc 0/0 00000100:00100000:1.0:1789762892.484074:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 700 00002000:00100000:1.0:1789762892.484083:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0xc:0x0] (3145728->4194303) req@ffff9c37d1ae0000 x1876702067468544/t4294967403(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762892.484092:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1ae0000 with x1876702067468544 ext(3145728->4194303) 00002000:00010000:1.0:1789762892.484097:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d1ae0000 00000100:00100000:3.0:1789762892.486863:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762892.486908:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067472640 00000100:00100000:0.0:1789762892.486929:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067472640 00002000:00100000:0.0:1789762892.486958:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0xd:0x0] (1048576->2097151) req@ffff9c37fdfe0700 x1876702067472640/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762892.486965:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfe0700 with x1876702067472640 ext(1048576->2097151) 00002000:00010000:0.0:1789762892.486969:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fdfe0700 00000100:00100000:0.0:1789762892.486981:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 1 deadline 334869342170 time 334869259177 state 0x0 ref 1 00000100:00100000:0.0:1789762892.486987:0:13301:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 8: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 1 deadline 334869342170 time 334869259177 state 0x0 ref 1 00000100:00100000:0.0:1789762892.487003:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 3 deadline 365784076977 time 365734076977 state 0x1 ref 1 00000100:00100000:0.0:1789762892.487305:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762892.487354:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067472512 00000100:00100000:1.0:1789762892.487360:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067472512 00002000:00100000:1.0:1789762892.487379:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xd:0x0] (3145728->4194303) req@ffff9c37c96a2680 x1876702067472512/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762892.487387:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c96a2680 with x1876702067472512 ext(3145728->4194303) 00002000:00010000:1.0:1789762892.487391:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c96a2680 00000100:00100000:1.0:1789762892.487400:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 3 deadline 365784076977 time 365734076977 state 0x1 ref 2 00000100:00100000:0.0:1789762892.489729:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762892.489791:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067472768 00000100:00100000:2.0:1789762892.489797:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067472768 00002000:00100000:2.0:1789762892.489817:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0xd:0x0] (0->1048575) req@ffff9c37c96a0700 x1876702067472768/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762892.489825:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c96a0700 with x1876702067472768 ext(0->1048575) 00010000:00010000:2.0:1789762892.489831:0:14170:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 24 pid: 8052 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.489844:0:14170:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 24 pid: 8052 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762892.489850:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9c37c96a0700 00000100:00100000:2.0:1789762892.489867:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763200 gen 0 rate 20 depth 3 token 2 deadline 365751159135 time 365701187388 state 0x0 ref 1 00000100:00100000:2.0:1789762892.489875:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763200 gen 0 rate 20 depth 3 token 2 deadline 365751187388 time 365701187388 state 0x1 ref 1 00000100:00100000:2.0:1789762892.489896:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763200 gen 0 rate 20 depth 3 token 1 deadline 365751187388 time 365736974958 state 0x0 ref 1 00000100:00100000:2.0:1789762892.489905:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:2397:x1876702067472768:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762892.494165:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2397:x1876702067472768:12345-192.168.203.37@tcp:4:dd.0 Request processed in 4246us (4422us total) trans 4294967404 rc 0/0 00000100:00100000:2.0:1789762892.494175:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 7 00000100:00100000:2.0:1789762892.494183:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763200 gen 0 rate 20 depth 3 token 1 deadline 365751187388 time 365736974958 state 0x2 ref 0 00002000:00100000:2.0:1789762892.494192:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0xd:0x0] (0->1048575) req@ffff9c37c96a0700 x1876702067472768/t4294967404(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762892.494199:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c96a0700 with x1876702067472768 ext(0->1048575) 00010000:00010000:2.0:1789762892.494205:0:14170:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 26 pid: 8052 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762892.494219:0:14170:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 26 pid: 8052 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762892.494225:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9c37c96a0700 00000100:00100000:3.0:1789762892.495967:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762892.496015:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067472896 00000100:00100000:1.0:1789762892.496022:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067472896 00000100:00100000:3.0:1789762892.496042:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00002000:00100000:1.0:1789762892.496048:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000400:0xd:0x0] (0->1048575) req@ffff9c37fdfe3800 x1876702067472896/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762892.496058:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfe3800 with x1876702067472896 ext(0->1048575) 00002000:00010000:1.0:1789762892.496062:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fdfe3800 00000100:00100000:1.0:1789762892.496072:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 3 deadline 365784076977 time 365734076977 state 0x1 ref 3 00000100:00100000:0.0:1789762892.496098:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067473024 00000100:00100000:0.0:1789762892.496103:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067473024 00010000:00010000:0.0:1789762892.496124:0:9878:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 26 pid: 8052 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762892.496150:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 84 00000100:00100000:0.0:1789762892.496167:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2397:x1876702067473024:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762892.496178:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1789762892.496189:0:9878:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0xd:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:0.0:1789762892.496192:0:9878:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0xd:0x0] updating lvb mtime from disk: 0 -> 1789762892 00002000:00010000:0.0:1789762892.496194:0:9878:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0xd:0x0] updating lvb ctime from disk: 0 -> 1789762892 00010000:00010000:0.0:1789762892.496198:0:9878:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 26 pid: 8052 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496214:0:9878:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 26 pid: 8052 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496233:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2bceba9/0xdf5a7ac59f1769cc lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c974 expref: 26 pid: 8052 timeout: 465 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496246:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496250:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496256:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f1769d3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c97b expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496263:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f1769d3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c97b expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496270:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496272:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496276:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f1769da lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c982 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496281:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f1769da lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c982 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496285:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496287:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496290:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f28035ec/0xdf5a7ac59f1769ef lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c997 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496299:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f28035ec/0xdf5a7ac59f1769ef lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c997 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496304:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496306:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496309:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002f037a74/0xdf5a7ac59f1769fd lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9a5 expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496314:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002f037a74/0xdf5a7ac59f1769fd lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9a5 expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496319:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496320:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496323:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000027c972f9/0xdf5a7ac59f176a04 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9ac expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496328:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000027c972f9/0xdf5a7ac59f176a04 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9ac expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496332:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496334:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496337:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005926da61/0xdf5a7ac59f176a2e lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9d6 expref: 25 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496342:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005926da61/0xdf5a7ac59f176a2e lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9d6 expref: 25 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496352:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496354:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762892.496357:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000054047fbc/0xdf5a7ac59f176a35 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9dd expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496362:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000054047fbc/0xdf5a7ac59f176a35 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9dd expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762892.496386:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000054047fbc/0xdf5a7ac59f176a35 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9dd expref: 25 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762892.496395:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074716544:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762892.496450:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005926da61/0xdf5a7ac59f176a2e lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9d6 expref: 25 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762892.496456:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074716672:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762892.496475:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000027c972f9/0xdf5a7ac59f176a04 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9ac expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762892.496482:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074716800:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762892.496502:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002f037a74/0xdf5a7ac59f1769fd lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9a5 expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762892.496529:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a3100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074716928:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762892.496560:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f28035ec/0xdf5a7ac59f1769ef lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c997 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762892.496571:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a1500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074717056:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762892.496616:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f1769da lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c982 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762892.496628:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074717184:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762892.496655:0:9878:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f1769d3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c97b expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762892.496663:0:9878:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074717312:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762892.496686:0:9878:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5b1cd80 going to sleep for 11 seconds 00000100:00100000:0.0:1789762892.498673:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074716544:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762892.498688:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074716672:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762892.498697:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074716800:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762892.498704:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a3100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074716928:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762892.498710:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a1500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074717056:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762892.498714:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074717184:192.168.203.37@tcp:105: 00000100:00100000:0.0:1789762892.506424:0:9878:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:9878:1876702074717312:192.168.203.37@tcp:105: 00010000:00010000:0.0:1789762892.506446:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762892.506472:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2397:x1876702067473024:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 10317us (10431us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762892.506478:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 84 00000100:00100000:2.0:1789762892.522174:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.522221:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067473152 00000100:00100000:3.0:1789762892.522227:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067473152 00002000:00100000:3.0:1789762892.522258:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0xd:0x0] (7340032->8388607) req@ffff9c37d8cf5f80 x1876702067473152/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762892.522267:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf5f80 with x1876702067473152 ext(7340032->8388607) 00002000:00010000:3.0:1789762892.522270:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf5f80 00000100:00100000:3.0:1789762892.522282:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 3 deadline 365784076977 time 365734076977 state 0x1 ref 4 00000100:00100000:1.0:1789762892.523959:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762892.524003:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067473280 00000100:00100000:0.0:1789762892.524009:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067473280 00002000:00100000:0.0:1789762892.524029:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000400:0xd:0x0] (5242880->6291455) req@ffff9c37f6089880 x1876702067473280/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762892.524036:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f6089880 with x1876702067473280 ext(5242880->6291455) 00002000:00010000:0.0:1789762892.524040:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37f6089880 00000100:00100000:0.0:1789762892.524049:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 3 deadline 365784076977 time 365734076977 state 0x1 ref 5 00000100:00100000:3.0:1789762892.530202:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762892.530279:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067473408 00000100:00100000:2.0:1789762892.530299:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067473408 00002000:00100000:2.0:1789762892.530337:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0xd:0x0] (5242880->6291455) req@ffff9c37fdfe2680 x1876702067473408/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762892.530346:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfe2680 with x1876702067473408 ext(5242880->6291455) 00002000:00010000:2.0:1789762892.530350:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fdfe2680 00000100:00100000:2.0:1789762892.530364:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 3 deadline 365784076977 time 365734076977 state 0x1 ref 6 00000100:00100000:2.0:1789762892.535534:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.535573:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067473536 00000100:00100000:3.0:1789762892.535581:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067473536 00002000:00100000:3.0:1789762892.535610:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0xd:0x0] (7340032->8388607) req@ffff9c37d8cf7480 x1876702067473536/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762892.535641:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf7480 with x1876702067473536 ext(7340032->8388607) 00002000:00010000:3.0:1789762892.535646:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf7480 00000100:00100000:3.0:1789762892.535660:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 3 deadline 365784076977 time 365734076977 state 0x1 ref 7 00000100:00100000:1.0:1789762892.537086:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 2 deadline 365834156084 time 365784156084 state 0x1 ref 7 00000100:00100000:1.0:1789762892.537106:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2398:x1876702067472640:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762892.538702:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762892.538766:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067473664 00000100:00100000:0.0:1789762892.538773:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067473664 00002000:00100000:0.0:1789762892.538796:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0xd:0x0] (2097152->3145727) req@ffff9c37d8cf5c00 x1876702067473664/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762892.538804:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf5c00 with x1876702067473664 ext(2097152->3145727) 00002000:00010000:0.0:1789762892.538808:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf5c00 00000100:00100000:0.0:1789762892.538818:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 2 deadline 365834156084 time 365784156084 state 0x1 ref 8 00000100:00100000:1.0:1789762892.539727:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:2398:x1876702067472640:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2625us (52864us total) trans 4294967918 rc 0/0 00000100:00100000:1.0:1789762892.539735:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 709 00002000:00100000:1.0:1789762892.539742:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xd:0x0] (1048576->2097151) req@ffff9c37fdfe0700 x1876702067472640/t4294967918(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762892.539749:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfe0700 with x1876702067472640 ext(1048576->2097151) 00002000:00010000:1.0:1789762892.539753:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fdfe0700 00000100:00100000:2.0:1789762892.541758:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762892.541805:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067473792 00000100:00100000:0.0:1789762892.541810:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067473792 00002000:00100000:0.0:1789762892.541829:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0xd:0x0] (6291456->7340031) req@ffff9c37d8cf6d80 x1876702067473792/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762892.541841:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf6d80 with x1876702067473792 ext(6291456->7340031) 00002000:00010000:0.0:1789762892.541844:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf6d80 00000100:00100000:0.0:1789762892.541853:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 2 deadline 365834156084 time 365784156084 state 0x1 ref 8 00000100:00100000:3.0:1789762892.577542:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 365924607371 time 365824607371 state 0x1 ref 8 00000100:00100000:3.0:1789762892.577576:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1aba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:2398:x1876702067468672:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762892.580441:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1aba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2398:x1876702067468672:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2869us (363244us total) trans 4294967919 rc 0/0 00000100:00100000:3.0:1789762892.580449:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 701 00002000:00100000:3.0:1789762892.580459:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0xc:0x0] (4194304->5242879) req@ffff9c37d1aba300 x1876702067468672/t4294967919(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762892.580468:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1aba300 with x1876702067468672 ext(4194304->5242879) 00002000:00010000:3.0:1789762892.580473:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d1aba300 00000100:00100000:3.0:1789762892.580499:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 1 deadline 365877575537 time 365827575537 state 0x1 ref 8 00000100:00100000:3.0:1789762892.580509:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2398:x1876702067472512:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762892.582699:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2398:x1876702067472512:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2189us (95391us total) trans 4294967920 rc 0/0 00000100:00100000:3.0:1789762892.582710:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 710 00002000:00100000:3.0:1789762892.582719:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0xd:0x0] (3145728->4194303) req@ffff9c37c96a2680 x1876702067472512/t4294967920(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762892.582730:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c96a2680 with x1876702067472512 ext(3145728->4194303) 00002000:00010000:3.0:1789762892.582736:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c96a2680 00000100:00100000:3.0:1789762892.582763:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 365879839817 time 365829839817 state 0x1 ref 7 00000100:00100000:3.0:1789762892.582774:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2398:x1876702067472896:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762892.582970:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762892.583017:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067473920 00000100:00100000:1.0:1789762892.583027:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067473920 00002000:00100000:1.0:1789762892.583062:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0xd:0x0] (4194304->5242879) req@ffff9c37c96a3b80 x1876702067473920/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762892.583080:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c96a3b80 with x1876702067473920 ext(4194304->5242879) 00002000:00010000:1.0:1789762892.583084:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c96a3b80 00000100:00100000:1.0:1789762892.583099:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 365879839817 time 365829839817 state 0x1 ref 8 00000100:00100000:3.0:1789762892.585047:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2398:x1876702067472896:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2273us (89080us total) trans 4294967921 rc 0/0 00000100:00100000:3.0:1789762892.585054:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 711 00002000:00100000:3.0:1789762892.585061:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000400:0xd:0x0] (0->1048575) req@ffff9c37fdfe3800 x1876702067472896/t4294967921(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762892.585068:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfe3800 with x1876702067472896 ext(0->1048575) 00002000:00010000:3.0:1789762892.585071:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fdfe3800 00000100:00100000:1.0:1789762892.632836:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 365929900141 time 365879900141 state 0x1 ref 7 00000100:00100000:1.0:1789762892.632860:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2396:x1876702067473152:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762892.635738:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2396:x1876702067473152:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2884us (113563us total) trans 4294967922 rc 0/0 00000100:00100000:1.0:1789762892.635750:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 712 00002000:00100000:1.0:1789762892.635764:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000400:0xd:0x0] (7340032->8388607) req@ffff9c37d8cf5f80 x1876702067473152/t4294967922(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762892.635774:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf5f80 with x1876702067473152 ext(7340032->8388607) 00002000:00010000:1.0:1789762892.635779:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf5f80 00000100:00100000:1.0:1789762892.677620:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 366024685330 time 365924685330 state 0x1 ref 7 00000100:00100000:1.0:1789762892.677644:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2398:x1876702067468800:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762892.680309:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2398:x1876702067468800:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2675us (456468us total) trans 4294967923 rc 0/0 00000100:00100000:1.0:1789762892.680317:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 702 00002000:00100000:1.0:1789762892.680326:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xc:0x0] (5242880->6291455) req@ffff9c37ff69b480 x1876702067468800/t4294967923(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762892.680339:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37ff69b480 with x1876702067468800 ext(5242880->6291455) 00002000:00010000:1.0:1789762892.680344:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37ff69b480 00000100:00100000:0.0:1789762892.682897:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 365979970962 time 365929970962 state 0x1 ref 6 00000100:00100000:0.0:1789762892.682911:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2398:x1876702067473280:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762892.685149:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2398:x1876702067473280:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2239us (161191us total) trans 4294967924 rc 0/0 00000100:00100000:0.0:1789762892.685172:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 713 00002000:00100000:0.0:1789762892.685182:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0xd:0x0] (5242880->6291455) req@ffff9c37f6089880 x1876702067473280/t4294967924(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762892.685192:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f6089880 with x1876702067473280 ext(5242880->6291455) 00002000:00010000:0.0:1789762892.685198:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37f6089880 00000100:00100000:1.0:1789762892.733009:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366030067823 time 365980067823 state 0x1 ref 5 00000100:00100000:1.0:1789762892.733041:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2397:x1876702067473408:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762892.736125:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:2397:x1876702067473408:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3095us (205944us total) trans 4294967405 rc 0/0 00000100:00100000:3.0:1789762892.736135:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 714 00002000:00100000:3.0:1789762892.736145:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0xd:0x0] (5242880->6291455) req@ffff9c37fdfe2680 x1876702067473408/t4294967405(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762892.736167:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfe2680 with x1876702067473408 ext(5242880->6291455) 00002000:00010000:3.0:1789762892.736173:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fdfe2680 00000100:00100000:0.0:1789762892.740058:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762892.740083:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762892.740103:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067474176 00000100:00100000:1.0:1789762892.740112:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067474176 00000100:00100000:3.0:1789762892.740134:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067474048 00000100:00100000:3.0:1789762892.740139:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067474048 00002000:00100000:1.0:1789762892.740147:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0xd:0x0] (3145728->4194303) req@ffff9c37c96a2a00 x1876702067474176/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762892.740166:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c96a2a00 with x1876702067474176 ext(3145728->4194303) 00002000:00010000:1.0:1789762892.740170:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c96a2a00 00002000:00100000:3.0:1789762892.740174:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0xd:0x0] (6291456->7340031) req@ffff9c37c96a1500 x1876702067474048/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762892.740184:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c96a1500 with x1876702067474048 ext(6291456->7340031) 00000100:00100000:1.0:1789762892.740191:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366030067823 time 365980067823 state 0x1 ref 5 00002000:00010000:3.0:1789762892.740197:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c96a1500 00000100:00100000:3.0:1789762892.740209:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366030067823 time 365980067823 state 0x1 ref 6 00000100:00100000:1.0:1789762892.777701:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 366124765495 time 366024765495 state 0x1 ref 6 00000100:00100000:1.0:1789762892.777725:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2398:x1876702067468928:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762892.780968:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2398:x1876702067468928:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3249us (550543us total) trans 4294967925 rc 0/0 00000100:00100000:0.0:1789762892.780979:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 703 00002000:00100000:0.0:1789762892.780990:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xc:0x0] (6291456->7340031) req@ffff9c37d1ae3480 x1876702067468928/t4294967925(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762892.781010:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1ae3480 with x1876702067468928 ext(6291456->7340031) 00002000:00010000:0.0:1789762892.781016:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d1ae3480 00000100:00100000:0.0:1789762892.783052:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366080122810 time 366030122810 state 0x1 ref 6 00000100:00100000:0.0:1789762892.783069:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:2398:x1876702067473536:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762892.785386:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2398:x1876702067473536:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2319us (249852us total) trans 4294967406 rc 0/0 00000100:00100000:0.0:1789762892.785393:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 715 00002000:00100000:0.0:1789762892.785401:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0xd:0x0] (7340032->8388607) req@ffff9c37d8cf7480 x1876702067473536/t4294967406(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762892.785408:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf7480 with x1876702067473536 ext(7340032->8388607) 00002000:00010000:0.0:1789762892.785412:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf7480 00000100:00100000:1.0:1789762892.787614:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762892.787678:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067474560 00000100:00100000:0.0:1789762892.787685:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067474560 00000100:00100000:0.0:1789762892.787727:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 86 00000100:00100000:0.0:1789762892.787735:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:15134:x1876702067474560:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:0.0:1789762892.788020:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:15134:x1876702067474560:12345-192.168.203.37@tcp:35:dd.0 Request processed in 284us (406us total) trans 4294967432 rc 0/0 00000100:00100000:2.0:1789762892.788032:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762892.788037:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 86 00000100:00100000:3.0:1789762892.788095:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067474304 00000100:00100000:3.0:1789762892.788101:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067474304 00002000:00100000:3.0:1789762892.788188:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0xd:0x0] (4194304->5242879) req@ffff9c37d8cf5880 x1876702067474304/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762892.788199:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf5880 with x1876702067474304 ext(4194304->5242879) 00002000:00010000:3.0:1789762892.788204:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf5880 00000100:00100000:3.0:1789762892.788219:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366080122810 time 366030122810 state 0x1 ref 6 00000100:00100000:1.0:1789762892.788819:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762892.788864:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067474688 00000100:00100000:3.0:1789762892.788870:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067474688 00002000:00100000:3.0:1789762892.788892:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0xd:0x0] (1048576->2097151) req@ffff9c37f608ad80 x1876702067474688/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/0 e 0 to 0 dl 1789762903 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762892.788904:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f608ad80 with x1876702067474688 ext(1048576->2097151) 00002000:00010000:3.0:1789762892.788908:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f608ad80 00000100:00100000:3.0:1789762892.788940:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366080122810 time 366030122810 state 0x1 ref 7 00000100:00100000:0.0:1789762892.833116:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366130183636 time 366080183636 state 0x1 ref 7 00000100:00100000:0.0:1789762892.833134:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:2397:x1876702067473664:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762892.835788:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2397:x1876702067473664:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2657us (297086us total) trans 4294967407 rc 0/0 00000100:00100000:0.0:1789762892.835798:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 716 00002000:00100000:0.0:1789762892.835810:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0xd:0x0] (2097152->3145727) req@ffff9c37d8cf5c00 x1876702067473664/t4294967407(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762892.835821:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf5c00 with x1876702067473664 ext(2097152->3145727) 00002000:00010000:0.0:1789762892.835828:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf5c00 00000100:00100000:0.0:1789762892.877749:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 366224815181 time 366124815181 state 0x1 ref 5 00000100:00100000:0.0:1789762892.877772:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2398:x1876702067469056:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762892.883580:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1abad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:2398:x1876702067469056:12345-192.168.203.37@tcp:4:dd.500 Request processed in 5815us (646424us total) trans 4294967926 rc 0/0 00000100:00100000:0.0:1789762892.883588:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 704 00002000:00100000:0.0:1789762892.883597:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xc:0x0] (7340032->8388607) req@ffff9c37d1abad80 x1876702067469056/t4294967926(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762892.883606:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1abad80 with x1876702067469056 ext(7340032->8388607) 00002000:00010000:0.0:1789762892.883612:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d1abad80 00000100:00100000:0.0:1789762892.883636:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366180713115 time 366130713115 state 0x1 ref 6 00000100:00100000:0.0:1789762892.883646:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2397:x1876702067473792:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762892.885769:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+44:2397:x1876702067473792:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2123us (344010us total) trans 4294967927 rc 0/0 00000100:00100000:0.0:1789762892.885780:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 717 00002000:00100000:0.0:1789762892.885787:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000400:0xd:0x0] (6291456->7340031) req@ffff9c37d8cf6d80 x1876702067473792/t4294967927(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762892.885795:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf6d80 with x1876702067473792 ext(6291456->7340031) 00002000:00010000:0.0:1789762892.885799:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d8cf6d80 00000100:00100000:0.0:1789762892.933690:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366230756783 time 366180756783 state 0x1 ref 5 00000100:00100000:0.0:1789762892.933718:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:2397:x1876702067473920:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762892.936704:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+45:2397:x1876702067473920:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3001us (353736us total) trans 4294967928 rc 0/0 00000100:00100000:0.0:1789762892.936712:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 718 00002000:00100000:0.0:1789762892.936721:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000400:0xd:0x0] (4194304->5242879) req@ffff9c37c96a3b80 x1876702067473920/t4294967928(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762892.936730:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c96a3b80 with x1876702067473920 ext(4194304->5242879) 00002000:00010000:0.0:1789762892.936735:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c96a3b80 00000100:00100000:3.0:1789762892.977832:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 366324893318 time 366224893318 state 0x1 ref 4 00000100:00100000:3.0:1789762892.977859:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2398:x1876702067469184:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762892.980994:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2398:x1876702067469184:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3142us (698751us total) trans 4294967408 rc 0/0 00000100:00100000:3.0:1789762892.981004:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 705 00002000:00100000:3.0:1789762892.981014:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0xc:0x0] (4194304->5242879) req@ffff9c37fd9f6d80 x1876702067469184/t4294967408(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762892.981023:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fd9f6d80 with x1876702067469184 ext(4194304->5242879) 00002000:00010000:3.0:1789762892.981030:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fd9f6d80 00000100:00100000:1.0:1789762892.983749:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366280822824 time 366230822824 state 0x1 ref 4 00000100:00100000:1.0:1789762892.983764:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2397:x1876702067474176:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762892.986208:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2397:x1876702067474176:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2446us (246151us total) trans 4294967409 rc 0/0 00000100:00100000:0.0:1789762892.986216:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 719 00002000:00100000:0.0:1789762892.986223:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000400:0xd:0x0] (3145728->4194303) req@ffff9c37c96a2a00 x1876702067474176/t4294967409(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762892.986230:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c96a2a00 with x1876702067474176 ext(3145728->4194303) 00002000:00010000:0.0:1789762892.986234:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c96a2a00 00000100:00100000:3.0:1789762893.033840:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366330905946 time 366280905946 state 0x1 ref 3 00000100:00100000:3.0:1789762893.033878:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2397:x1876702067474048:12345-192.168.203.37@tcp:4:dd.0 00000100:00080000:0.0:1789762893.033898:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762893.037019:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2397:x1876702067474048:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3161us (296933us total) trans 4294967410 rc 0/0 00000100:00100000:3.0:1789762893.037028:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 720 00002000:00100000:3.0:1789762893.037038:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000400:0xd:0x0] (6291456->7340031) req@ffff9c37c96a1500 x1876702067474048/t4294967410(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762893.037049:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c96a1500 with x1876702067474048 ext(6291456->7340031) 00002000:00010000:3.0:1789762893.037054:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c96a1500 00000100:00100000:1.0:1789762893.077886:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 366424950455 time 366324950455 state 0x1 ref 3 00000100:00100000:1.0:1789762893.077909:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2398:x1876702067469312:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762893.080881:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2398:x1876702067469312:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2978us (791683us total) trans 4294967411 rc 0/0 00000100:00100000:3.0:1789762893.080892:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 706 00002000:00100000:3.0:1789762893.080902:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000400:0xc:0x0] (5242880->6291455) req@ffff9c37d1ae1c00 x1876702067469312/t4294967411(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762893.080910:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d1ae1c00 with x1876702067469312 ext(5242880->6291455) 00002000:00010000:3.0:1789762893.080933:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d1ae1c00 00000100:00100000:0.0:1789762893.083947:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366381019739 time 366331019739 state 0x1 ref 2 00000100:00100000:0.0:1789762893.083962:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2395:x1876702067474304:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762893.086274:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2395:x1876702067474304:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2313us (298244us total) trans 4294967412 rc 0/0 00000100:00100000:3.0:1789762893.086281:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 721 00002000:00100000:3.0:1789762893.086289:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000400:0xd:0x0] (4194304->5242879) req@ffff9c37d8cf5880 x1876702067474304/t4294967412(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762893.086296:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d8cf5880 with x1876702067474304 ext(4194304->5242879) 00002000:00010000:3.0:1789762893.086306:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d8cf5880 00000100:00100000:0.0:1789762893.134055:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366381019739 time 366381124498 state 0x0 ref 1 00000100:00100000:0.0:1789762893.134072:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2395:x1876702067474688:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762893.139785:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2395:x1876702067474688:12345-192.168.203.37@tcp:4:dd.0 Request processed in 5715us (350964us total) trans 4294967413 rc 0/0 00000100:00100000:2.0:1789762893.139793:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 722 00000100:00100000:2.0:1789762893.139802:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366381019739 time 366381124498 state 0x2 ref 0 00002000:00100000:2.0:1789762893.139815:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000400:0xd:0x0] (1048576->2097151) req@ffff9c37f608ad80 x1876702067474688/t4294967413(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762893.139826:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f608ad80 with x1876702067474688 ext(1048576->2097151) 00002000:00010000:2.0:1789762893.139832:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f608ad80 00000100:00100000:0.0:1789762893.178016:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 366525082800 time 366425082800 state 0x1 ref 2 00000100:00100000:0.0:1789762893.178040:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2398:x1876702067469440:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762893.180613:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2398:x1876702067469440:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2581us (797615us total) trans 4294967414 rc 0/0 00000100:00100000:0.0:1789762893.180621:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 707 00002000:00100000:0.0:1789762893.180631:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000400:0xc:0x0] (6291456->7340031) req@ffff9c37fdfe1c00 x1876702067469440/t4294967414(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762893.180640:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fdfe1c00 with x1876702067469440 ext(6291456->7340031) 00002000:00010000:0.0:1789762893.180645:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fdfe1c00 00000100:00100000:3.0:1789762893.278127:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 366525082800 time 366525194411 state 0x0 ref 1 00000100:00100000:3.0:1789762893.278162:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2398:x1876702067469568:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762893.281137:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:2398:x1876702067469568:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2995us (891412us total) trans 4294967415 rc 0/0 00000100:00100000:3.0:1789762893.281144:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 708 00000100:00100000:3.0:1789762893.281166:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 366525082800 time 366525194411 state 0x2 ref 0 00002000:00100000:3.0:1789762893.281177:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000400:0xc:0x0] (7340032->8388607) req@ffff9c37fd9f4e00 x1876702067469568/t4294967415(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:673/0 lens 488/448 e 0 to 0 dl 1789762903 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762893.281185:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fd9f4e00 with x1876702067469568 ext(7340032->8388607) 00002000:00010000:3.0:1789762893.281190:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37fd9f4e00 00000100:00100000:3.0:1789762893.312037:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074717440:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762893.312116:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762893.312173:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074717440 00000100:00100000:1.0:1789762893.312180:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074717440 00000100:00100000:1.0:1789762893.312222:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:1.0:1789762893.312231:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074717440:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789762893.312258:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:1.0:1789762893.312304:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074717440:12345-0@lo:41:osp-pre-0-1.0 Request processed in 74us (188us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762893.312309:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00100000:3.0:1789762893.312351:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074717440:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762893.318896:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762893.318982:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067474816 00000100:00100000:1.0:1789762893.318988:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067474816 00000100:00100000:0.0:1789762893.319007:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762893.319032:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067474944 00000100:00100000:1.0:1789762893.319036:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067474944 00000100:00100000:3.0:1789762893.319059:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 59 00000100:00100000:2.0:1789762893.319068:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 60 00000100:00080000:0.0:1789762893.319071:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762893.319073:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:6357:x1876702067474816:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762893.319085:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 8 locks, starting at 0 00000100:00100000:2.0:1789762893.319086:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c91050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+45:6823:x1876702067474944:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762893.319098:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 8 locks, starting at 0 00002000:00010000:3.0:1789762893.319107:0:8070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0xc:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:2.0:1789762893.319111:0:6405:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xc:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:3.0:1789762893.319113:0:8070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000400:0xc:0x0] updating lvb mtime from disk: 0 -> 1789762892 00002000:00010000:3.0:1789762893.319116:0:8070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000400:0xc:0x0] updating lvb ctime from disk: 0 -> 1789762892 00002000:00010000:3.0:1789762893.319118:0:8070:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000400:0xc:0x0] updating lvb blocks from disk: 1 -> 2051 00002000:00010000:2.0:1789762893.319122:0:6405:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xc:0x0] updating lvb mtime from disk: 0 -> 1789762892 00002000:00010000:2.0:1789762893.319124:0:6405:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xc:0x0] updating lvb ctime from disk: 0 -> 1789762892 00010000:00010000:2.0:1789762893.319160:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f176994 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c95f expref: 45 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.319161:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000496311b/0xdf5a7ac59f1769a2 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c966 expref: 43 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762893.319198:0:6405:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xd:0x0] updating lvb size from disk: 3145728 -> 8388608 00002000:00010000:3.0:1789762893.319206:0:8070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000400:0xd:0x0] updating lvb size from disk: 1048576 -> 8388608 00010000:00010000:2.0:1789762893.319210:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a19 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9c1 expref: 44 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.319224:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f1769da lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c982 expref: 42 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.319228:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176a12 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9ba expref: 43 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.319256:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e65ce32d/0xdf5a7ac59f176a0b lrc: 0/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 7 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9b3 expref: 42 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.319266:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005926da61/0xdf5a7ac59f176a2e lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 8 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9d6 expref: 41 pid: 8052 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.319271:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000974c0faa/0xdf5a7ac59f176a27 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 6 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9cf expref: 41 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.319303:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f28035ec/0xdf5a7ac59f1769ef lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c997 expref: 40 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.319304:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f1769f6 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 5 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c99e expref: 40 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.319343:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f037a74/0xdf5a7ac59f1769fd lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9a5 expref: 39 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.319344:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f1769e8 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c990 expref: 39 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.319383:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002cfe4f3f/0xdf5a7ac59f1769d3 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c97b expref: 38 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.319384:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb2b668/0xdf5a7ac59f176a20 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9c8 expref: 38 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.319411:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000054047fbc/0xdf5a7ac59f176a35 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 4 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9dd expref: 37 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.319426:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762893.319428:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000027c972f9/0xdf5a7ac59f176a04 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9ac expref: 36 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.319443:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762893.319461:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c91050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:6823:x1876702067474944:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 380us (456us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762893.319468:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 60 00000100:00100000:3.0:1789762893.319474:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:6357:x1876702067474816:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 406us (578us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762893.319483:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 59 00000100:00100000:2.0:1789762893.350591:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762893.350622:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762893.350654:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067475200 00000100:00100000:3.0:1789762893.350663:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067475200 00000100:00100000:0.0:1789762893.350691:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067475456 00000100:00100000:0.0:1789762893.350699:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067475456 00000100:00080000:1.0:1789762893.350730:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762893.350761:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 308 00000100:00100000:0.0:1789762893.350784:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:15163:x1876702067475456:12345-192.168.203.37@tcp:101:dd.0 00000100:00100000:3.0:1789762893.350789:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 309 00000100:00100000:3.0:1789762893.350804:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:15162:x1876702067475200:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762893.350817:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762893.350818:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762893.350835:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af7e00/0xdf5a7ac59f176a3c lrc: 2/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9e4 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.350836:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176a43 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9eb expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.350951:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af6000/0xdf5a7ac59f176a4a lrc: 3/1,0 mode: --/PR res: [0x240000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.350963:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af6000/0xdf5a7ac59f176a4a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.350969:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176a51 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.350974:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af6000/0xdf5a7ac59f176a4a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.350986:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176a51 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762893.350987:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15:0x0]: returning size 16777216 00010000:00010000:0.0:1789762893.351000:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176a51 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762893.351010:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15:0x0]: returning size 16777216 00010000:00010000:3.0:1789762893.351121:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af7e00/0xdf5a7ac59f176a3c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9e4 expref: 29 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.351138:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176a43 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9eb expref: 29 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.351138:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af6000/0xdf5a7ac59f176a4a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9e4 expref: 29 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.351165:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:3.0:1789762893.351170:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762893.351183:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36c9af6000, rc 0) 00010000:00010000:0.0:1789762893.351195:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176a51 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9eb expref: 28 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.351205:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:0.0:1789762893.351209:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762893.351212:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca315600, rc 0) 00000100:00100000:3.0:1789762893.351231:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:15162:x1876702067475200:12345-192.168.203.37@tcp:101:dd.0 Request processed in 431us (647us total) trans 4294967433 rc 0/0 00000100:00100000:3.0:1789762893.351237:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 309 00000100:00100000:0.0:1789762893.351246:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:15163:x1876702067475456:12345-192.168.203.37@tcp:101:dd.0 Request processed in 465us (621us total) trans 4294967434 rc 0/0 00000100:00100000:0.0:1789762893.351255:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 308 00000100:00100000:1.0:1789762893.351566:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.351608:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067475712 00000100:00100000:0.0:1789762893.351614:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067475712 00000100:00100000:0.0:1789762893.351644:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 310 00000100:00100000:0.0:1789762893.351665:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:15164:x1876702067475712:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762893.351676:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762893.351685:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176a66 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9f2 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.351726:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176a6d lrc: 3/1,0 mode: --/PR res: [0x240000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.351737:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176a6d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.351747:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176a6d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762893.351766:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15:0x0]: returning size 16777216 00010000:00010000:0.0:1789762893.351841:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176a66 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9f2 expref: 30 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.351857:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176a6d lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9f2 expref: 29 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.351864:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:0.0:1789762893.351867:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762893.351870:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca314800, rc 0) 00000100:00100000:0.0:1789762893.351901:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:15164:x1876702067475712:12345-192.168.203.37@tcp:101:dd.0 Request processed in 238us (337us total) trans 4294967435 rc 0/0 00000100:00100000:0.0:1789762893.351909:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 310 00000100:00100000:2.0:1789762893.352963:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762893.352988:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762893.352992:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067476224 00000100:00100000:3.0:1789762893.353000:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067476224 00000100:00100000:2.0:1789762893.353012:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762893.353029:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067475840 00000100:00100000:1.0:1789762893.353036:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067475840 00000100:00100000:3.0:1789762893.353040:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 87 00000100:00100000:2.0:1789762893.353047:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.353054:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067476096 00000100:00100000:3.0:1789762893.353054:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:15162:x1876702067476224:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:0.0:1789762893.353062:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067476096 00000100:00100000:2.0:1789762893.353073:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067476352 00000100:00100000:2.0:1789762893.353079:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067476352 00000100:00100000:1.0:1789762893.353085:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 61 00000100:00100000:1.0:1789762893.353100:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2395:x1876702067475840:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762893.353113:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1789762893.353118:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 62 00000100:00100000:2.0:1789762893.353119:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 195 00010000:00010000:1.0:1789762893.353132:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176a51 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9eb expref: 32 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762893.353144:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2395:x1876702067476096:12345-192.168.203.37@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1789762893.353144:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:15163:x1876702067476352:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762893.353164:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:0.0:1789762893.353169:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762893.353171:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762893.353179:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1789762893.353185:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:15162:x1876702067476224:12345-192.168.203.37@tcp:35:dd.0 Request processed in 134us (222us total) trans 4294967436 rc 0/0 00000100:00100000:3.0:1789762893.353193:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 87 00010000:00010000:0.0:1789762893.353197:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af6000/0xdf5a7ac59f176a4a lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9e4 expref: 31 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.353198:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a7b lrc: 2/0,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9c9f9 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762893.353200:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2395:x1876702067475840:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 104us (211us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762893.353220:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 61 00010000:00010000:0.0:1789762893.353233:0:9878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:0.0:1789762893.353239:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00002000:00010000:2.0:1789762893.353241:0:9037:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xc:0x0] initial LVB size: 8388608, mtime: 0x6aad9d4c, atime: 0x0, ctime: 0x6aad9d4c, blocks: 0x1 00010000:00010000:2.0:1789762893.353260:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=366s v=(0 0) 00010000:00010000:2.0:1789762893.353266:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00000100:00100000:0.0:1789762893.353269:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2395:x1876702067476096:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 138us (256us total) trans 0 rc 0/0 00010000:00010000:2.0:1789762893.353279:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a7b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9f9 expref: 38 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762893.353282:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 62 00010000:00010000:2.0:1789762893.353303:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a7b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9f9 expref: 38 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.353326:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d1768000, rc 0) 00000100:00100000:2.0:1789762893.353354:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:15163:x1876702067476352:12345-192.168.203.37@tcp:101:dd.0 Request processed in 225us (311us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762893.353359:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 195 00000100:00100000:3.0:1789762893.353455:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762893.353494:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067476480 00000100:00100000:2.0:1789762893.353499:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067476480 00000100:00100000:2.0:1789762893.353530:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 63 00000100:00100000:2.0:1789762893.353536:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:2398:x1876702067476480:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762893.353543:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762893.353552:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176a6d lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9f2 expref: 28 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.353561:0:6405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:2.0:1789762893.353564:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762893.353587:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2398:x1876702067476480:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 52us (133us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762893.353592:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 63 00000100:00100000:1.0:1789762893.357377:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.357421:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067476736 00000100:00100000:0.0:1789762893.357427:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067476736 00000100:00100000:0.0:1789762893.357455:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 88 00000100:00100000:0.0:1789762893.357462:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbc450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:15164:x1876702067476736:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:0.0:1789762893.357552:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbc450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:15164:x1876702067476736:12345-192.168.203.37@tcp:35:dd.0 Request processed in 90us (176us total) trans 4294967437 rc 0/0 00000100:00100000:0.0:1789762893.357559:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 88 00000100:00100000:0.0:1789762893.357878:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762893.357911:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067476864 00000100:00100000:1.0:1789762893.357930:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067476864 00002000:00100000:1.0:1789762893.357957:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0xc:0x0] (0->1048575) req@ffff9c37c96a0a80 x1876702067476864/t0(0) o3->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:674/0 lens 488/0 e 0 to 0 dl 1789762904 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762893.357977:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c96a0a80 with x1876702067476864 ext(0->1048575) 00002000:00010000:1.0:1789762893.357980:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c96a0a80 00000100:00100000:1.0:1789762893.357993:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 335228189611 time 335228120661 state 0x0 ref 1 00000100:00100000:1.0:1789762893.357999:0:8058:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 8: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 335228189611 time 335228120661 state 0x0 ref 1 00000100:00100000:1.0:1789762893.358012:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 3 deadline 366705088325 time 366605088325 state 0x1 ref 1 00000100:00100000:1.0:1789762893.358029:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 366705088325 time 366605107931 state 0x0 ref 1 00000100:00100000:1.0:1789762893.358038:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2395:x1876702067476864:12345-192.168.203.37@tcp:3:dd.0 00000100:00100000:1.0:1789762893.361746:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2395:x1876702067476864:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3706us (3865us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762893.361754:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 723 00000100:00100000:1.0:1789762893.361761:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 366705088325 time 366605107931 state 0x2 ref 0 00002000:00100000:1.0:1789762893.361771:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000400:0xc:0x0] (0->1048575) req@ffff9c37c96a0a80 x1876702067476864/t0(0) o3->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:674/0 lens 488/440 e 0 to 0 dl 1789762904 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762893.361779:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c96a0a80 with x1876702067476864 ext(0->1048575) 00002000:00010000:1.0:1789762893.361782:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c96a0a80 00000100:00100000:3.0:1789762893.385436:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762893.385501:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067477120 00000100:00100000:2.0:1789762893.385509:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067477120 00000100:00100000:2.0:1789762893.385556:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 89 00000100:00100000:2.0:1789762893.385566:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+28:15176:x1876702067477120:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:2.0:1789762893.385760:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:15176:x1876702067477120:12345-192.168.203.37@tcp:35:dd.0 Request processed in 195us (328us total) trans 4294967438 rc 0/0 00000100:00100000:2.0:1789762893.385771:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 89 00000100:00100000:2.0:1789762893.386825:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762893.386874:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067477376 00000100:00100000:1.0:1789762893.386880:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067477376 00000100:00100000:1.0:1789762893.386912:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 311 00000100:00100000:1.0:1789762893.386944:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:15174:x1876702067477376:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762893.386957:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762893.386974:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762c000/0xdf5a7ac59f176a82 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca00 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.387035:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762de00/0xdf5a7ac59f176a89 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.387047:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762de00/0xdf5a7ac59f176a89 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x15:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.387057:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762de00/0xdf5a7ac59f176a89 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x15:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762893.387066:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000004:00200000:1.0:1789762893.387069:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15:0x0]: returning size 16777216 00000100:00100000:0.0:1789762893.387117:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067477632 00000100:00100000:0.0:1789762893.387123:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067477632 00010000:00010000:1.0:1789762893.387163:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762c000/0xdf5a7ac59f176a82 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca00 expref: 33 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762893.387172:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 312 00000100:00100000:0.0:1789762893.387181:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:15168:x1876702067477632:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762893.387186:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762de00/0xdf5a7ac59f176a89 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x15:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca00 expref: 32 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.387192:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762893.387211:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:1.0:1789762893.387216:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:0.0:1789762893.387217:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176a97 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca07 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.387220:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c762de00, rc 0) 00000100:00100000:1.0:1789762893.387261:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:15174:x1876702067477376:12345-192.168.203.37@tcp:101:dd.0 Request processed in 342us (438us total) trans 4294967439 rc 0/0 00010000:00010000:0.0:1789762893.387266:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176a9e lrc: 3/1,0 mode: --/CR res: [0x240000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762893.387268:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 311 00010000:00010000:0.0:1789762893.387287:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176a9e lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x15:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.387295:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176a9e lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x15:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762893.387302:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15:0x0]: returning size 16777216 00010000:00010000:0.0:1789762893.387359:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176a97 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca07 expref: 34 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.387375:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176a9e lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x15:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca07 expref: 33 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.387384:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:0.0:1789762893.387387:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:0.0:1789762893.387390:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca315400, rc 0) 00000100:00100000:0.0:1789762893.387426:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:15168:x1876702067477632:12345-192.168.203.37@tcp:101:dd.0 Request processed in 245us (362us total) trans 4294967440 rc 0/0 00000100:00100000:0.0:1789762893.387434:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 312 00000100:00100000:3.0:1789762893.388435:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762893.388463:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.388471:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067478016 00000100:00100000:0.0:1789762893.388476:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067478016 00000100:00100000:1.0:1789762893.388500:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067477888 00000100:00100000:1.0:1789762893.388506:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067477888 00000100:00100000:0.0:1789762893.388507:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 90 00000100:00100000:0.0:1789762893.388519:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:15168:x1876702067478016:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:1.0:1789762893.388554:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 64 00000100:00100000:1.0:1789762893.388566:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2396:x1876702067477888:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762893.388578:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762893.388602:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176a9e lrc: 0/0,0 mode: --/CR res: [0x240000402:0x15:0x0].0x0 bits 0xd/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca07 expref: 35 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.388613:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:1.0:1789762893.388616:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762893.388631:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:15168:x1876702067478016:12345-192.168.203.37@tcp:35:dd.0 Request processed in 113us (194us total) trans 4294967441 rc 0/0 00000100:00100000:0.0:1789762893.388643:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 90 00000100:00100000:1.0:1789762893.388644:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2396:x1876702067477888:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 77us (175us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762893.388654:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 64 00000100:00100000:2.0:1789762893.400881:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762893.400948:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067478144 00000100:00100000:3.0:1789762893.400955:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067478144 00000100:00100000:3.0:1789762893.400989:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 313 00000100:00100000:3.0:1789762893.401000:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2528000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:15178:x1876702067478144:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:3.0:1789762893.401013:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762893.401028:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af7e00/0xdf5a7ac59f176aac lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca0e expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.401082:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af6200/0xdf5a7ac59f176ab3 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.401092:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af6200/0xdf5a7ac59f176ab3 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.401107:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af6200/0xdf5a7ac59f176ab3 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762893.401127:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14:0x0]: returning size 16384 00000004:00200000:3.0:1789762893.401130:0:11692:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762893.401140:0:11692:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af6200/0xdf5a7ac59f176ab3 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.401170:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af7e00/0xdf5a7ac59f176aac lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca0e expref: 35 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.401195:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af6200/0xdf5a7ac59f176ab3 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca0e expref: 34 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.401201:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762893.401203:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789762893.401205:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36c9af6200, rc 0) 00000100:00100000:3.0:1789762893.401238:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2528000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:15178:x1876702067478144:12345-192.168.203.37@tcp:101:rm.0 Request processed in 241us (360us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762893.401242:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 313 00000100:00100000:2.0:1789762893.402227:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762893.402260:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067478400 00000100:00100000:3.0:1789762893.402264:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067478400 00000100:00100000:3.0:1789762893.402288:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 314 00000100:00100000:3.0:1789762893.402293:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:15178:x1876702067478400:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:3.0:1789762893.402301:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762893.402309:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af7c00/0xdf5a7ac59f176aba lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca15 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762893.402351:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14:0x0]: returning size 16384 00010000:00010000:3.0:1789762893.402403:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af7c00/0xdf5a7ac59f176aba lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca15 expref: 36 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.402411:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37c72bad00) 00010000:00010000:3.0:1789762893.402413:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762893.402415:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af7c00/0xdf5a7ac59f176aba lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca15 expref: 36 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.402424:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36c9af7c00, rc 0) 00000100:00100000:3.0:1789762893.402450:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:15178:x1876702067478400:12345-192.168.203.37@tcp:101:rm.0 Request processed in 157us (225us total) trans 4294967442 rc 301/301 00000100:00100000:3.0:1789762893.402462:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 314 00000100:00100000:2.0:1789762893.403874:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762893.403905:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067478528 00000100:00100000:3.0:1789762893.403909:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067478528 00000100:00100000:3.0:1789762893.403947:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 91 00000100:00100000:3.0:1789762893.403952:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:15178:x1876702067478528:12345-192.168.203.37@tcp:37:rm.0 00000100:00100000:3.0:1789762893.405812:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:15178:x1876702067478528:12345-192.168.203.37@tcp:37:rm.0 Request processed in 1859us (1939us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762893.405818:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 91 00000100:00100000:2.0:1789762893.407081:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.407132:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067478784 00000100:00100000:0.0:1789762893.407138:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067478784 00000100:00100000:0.0:1789762893.407175:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 92 00000100:00100000:0.0:1789762893.407181:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:15178:x1876702067478784:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:0.0:1789762893.407273:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:15178:x1876702067478784:12345-192.168.203.37@tcp:35:rm.0 Request processed in 92us (193us total) trans 4294967443 rc 0/0 00000100:00100000:0.0:1789762893.407278:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 92 00000100:00100000:2.0:1789762893.408032:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762893.408070:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067479040 00000100:00100000:1.0:1789762893.408075:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067479040 00000100:00100000:1.0:1789762893.408103:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 315 00000100:00100000:1.0:1789762893.408110:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:15178:x1876702067479040:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:1.0:1789762893.408119:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762893.408128:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762c000/0xdf5a7ac59f176ac8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca1c expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.408182:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762d800/0xdf5a7ac59f176acf lrc: 3/1,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.408191:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762d800/0xdf5a7ac59f176acf lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x14:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.408197:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762d800/0xdf5a7ac59f176acf lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x14:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762893.408203:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14:0x0]: returning size 16384 00010000:00010000:1.0:1789762893.408253:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762c000/0xdf5a7ac59f176ac8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca1c expref: 39 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.408265:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762d800/0xdf5a7ac59f176acf lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x14:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca1c expref: 38 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.408271:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:1.0:1789762893.408273:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1789762893.408275:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c762d800, rc 0) 00000100:00100000:1.0:1789762893.408301:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:15178:x1876702067479040:12345-192.168.203.37@tcp:101:rm.0 Request processed in 192us (270us total) trans 4294967444 rc 0/0 00000100:00100000:1.0:1789762893.408306:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 315 00000100:00100000:2.0:1789762893.409658:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.409702:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067479168 00000100:00100000:0.0:1789762893.409709:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067479168 00000100:00080000:1.0:1789762893.409769:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762893.409773:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 316 00000100:00100000:0.0:1789762893.409787:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15178:x1876702067479168:12345-192.168.203.37@tcp:41:rm.0 00000004:00000024:0.0:1789762893.409839:0:6418:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7719616512 avail 7715422208 00000100:00100000:0.0:1789762893.409892:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15178:x1876702067479168:12345-192.168.203.37@tcp:41:rm.0 Request processed in 108us (245us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762893.409899:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 316 00000100:00100000:2.0:1789762893.411064:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.411095:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067479424 00000100:00100000:0.0:1789762893.411100:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067479424 00000100:00100000:0.0:1789762893.411127:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 317 00000100:00100000:0.0:1789762893.411133:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2528e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:15178:x1876702067479424:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:0.0:1789762893.411176:0:6420:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1789762893.411202:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769400/0xdf5a7ac59f1769be lrc: 0/0,0 mode: --/PR res: [0x240000402:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c96d expref: 38 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.411220:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16:0x0].0x0 (ffff9c37d3bee900) 00010000:00010000:0.0:1789762893.411238:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36c9af6200/0xdf5a7ac59f176ab3 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca0e expref: 37 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.411247:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:0.0:1789762893.411249:0:6420:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762893.411263:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176add lrc: 3/0,1 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.411273:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176add lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.411280:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176add lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.411287:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f176ae4 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.411292:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f176ae4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.411296:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f176ae4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.411373:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f176aeb lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.411379:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f176aeb lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.411383:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f176aeb lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762893.412050:0:6420:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].12, 64 off8960 00000004:00080000:0.0:1789762893.412059:0:6420:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 366659140207ns, next at 367659138364ns, rc = 0 00000040:00080000:0.0:1789762893.412108:0:6420:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].12, 64 off8960 00000004:00080000:0.0:1789762893.412113:0:6420:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 366659194850ns, next at 367659194094ns, rc = 0 00010000:00010000:0.0:1789762893.412173:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176af2 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412181:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176af2 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412189:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176af2 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412197:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176af2 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412204:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176af2 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412212:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176af2 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412230:0:6420:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762893.412244:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f176aeb lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412250:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f176aeb lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.412251:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762893.412259:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f176aeb lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.412261:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176af2 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412268:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.412274:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f176aeb lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.412275:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762893.412281:0:7217:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176af2 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412284:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f176aeb lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412294:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16:0x0].0x0 (ffff9c37e37a7900) 00010000:00010000:0.0:1789762893.412297:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762893.412300:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f176aeb lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412304:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314e00/0xdf5a7ac59f176aeb lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.412305:0:7217:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176af2 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.412323:0:7217:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16:0x0].0x0 (ffff9c37e37a7900) 00010000:00010000:0.0:1789762893.412324:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176add lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.412326:0:7217:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789762893.412332:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176af2 lrc: 1/0,0 mode: --/PW res: [0x240000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412334:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176add lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.412349:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315400/0xdf5a7ac59f176af2 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412350:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176add lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412369:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176add lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412374:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.412377:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176add lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412382:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176add lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.412383:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762893.412389:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762893.412392:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762893.412393:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762893.412396:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.412399:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176add lrc: 3/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412403:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176add lrc: 0/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762893.412410:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000509c5bf6 reply state 00000000c5fa98ed transno 4294967445 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f176ae4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412418:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f176ae4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.412422:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x2b0e463d (ffff9c37d3bee200) 00010000:00010000:0.0:1789762893.412423:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762893.412425:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762893.412460:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2528e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:15178:x1876702067479424:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1326us (1397us total) trans 4294967445 rc 0/0 00000100:00100000:0.0:1789762893.412465:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 317 00000100:00100000:2.0:1789762893.413557:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762893.413601:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067479680 00000100:00100000:3.0:1789762893.413606:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067479680 00000100:00100000:3.0:1789762893.413633:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 93 00000100:00100000:3.0:1789762893.413638:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:15178:x1876702067479680:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:3.0:1789762893.413741:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:15178:x1876702067479680:12345-192.168.203.37@tcp:35:rm.0 Request processed in 102us (184us total) trans 4294967446 rc 0/0 00000100:00100000:3.0:1789762893.413755:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 93 00000100:00100000:2.0:1789762893.414508:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.414542:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067479936 00000100:00100000:0.0:1789762893.414548:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067479936 00000100:00100000:0.0:1789762893.414584:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 318 00000100:00100000:0.0:1789762893.414594:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:15178:x1876702067479936:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:0.0:1789762893.414622:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1789762893.414635:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762de00/0xdf5a7ac59f176a89 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x15:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca00 expref: 39 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.414647:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769e00/0xdf5a7ac59f176986 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c958 expref: 38 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.414657:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:0.0:1789762893.414662:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762893.414670:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176af9 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.414677:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176af9 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.414683:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176af9 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.414692:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176b00 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x14:0x0].0x73550ecb bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.414700:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176b00 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.414708:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176b00 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.414758:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176b07 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.414768:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176b07 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.414773:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176b07 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762893.415103:0:6419:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].13, 64 off9024 00000040:00080000:0.0:1789762893.415115:0:6419:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].13, 64 off9024 00010000:00010000:0.0:1789762893.415165:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176b0e lrc: 3/0,1 mode: --/PW res: [0x240000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415173:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176b0e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415179:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176b0e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415183:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176b0e lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415190:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176b0e lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415197:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176b0e lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415214:0:6419:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762893.415227:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176b07 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.415231:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762893.415242:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176b07 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x15:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.415242:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176b0e lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415266:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176b07 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.415266:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762893.415271:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176b0e lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415274:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.415279:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176b07 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415286:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176b07 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x15:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.415297:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176b0e lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415300:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37e37a7f00) 00010000:00010000:0.0:1789762893.415303:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762893.415306:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176b07 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415313:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176b07 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.415322:0:6407:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15:0x0].0x0 (ffff9c37e37a7f00) 00010000:00010000:3.0:1789762893.415326:0:6407:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762893.415327:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176af9 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.415329:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176b0e lrc: 1/0,0 mode: --/PW res: [0x240000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415337:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176af9 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.415340:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314200/0xdf5a7ac59f176b0e lrc: 0/0,0 mode: --/PW res: [0x240000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415346:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176af9 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415353:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176af9 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.415356:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762893.415357:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762893.415360:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762893.415362:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176af9 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.415363:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762893.415370:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176af9 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415377:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:0.0:1789762893.415378:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.415380:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176af9 lrc: 3/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415387:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176af9 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762893.415394:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000018101f56 reply state 00000000288ceadb transno 4294967447 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176b00 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415400:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176b00 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.415405:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x73550ecb (ffff9c37d3bee300) 00010000:00010000:0.0:1789762893.415406:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762893.415408:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762893.415441:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:15178:x1876702067479936:12345-192.168.203.37@tcp:36:rm.0 Request processed in 849us (933us total) trans 4294967447 rc 0/0 00000100:00100000:0.0:1789762893.415446:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 318 00000100:00100000:2.0:1789762893.417418:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762893.417443:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.417451:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067480320 00000100:00100000:0.0:1789762893.417458:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067480320 00000100:00100000:1.0:1789762893.417488:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067480064 00000100:00100000:0.0:1789762893.417494:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 94 00000100:00100000:1.0:1789762893.417496:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067480064 00000100:00100000:0.0:1789762893.417503:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbcc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:15178:x1876702067480320:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:1.0:1789762893.417542:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 65 00000100:00100000:1.0:1789762893.417552:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067480064:12345-192.168.203.37@tcp:103:rm.0 00010000:00010000:1.0:1789762893.417561:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762893.417575:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315e00/0xdf5a7ac59f1768e5 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c920 expref: 7 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.417592:0:6404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37e37a7200) 00010000:00010000:1.0:1789762893.417599:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762893.417626:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbcc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:15178:x1876702067480320:12345-192.168.203.37@tcp:35:rm.0 Request processed in 122us (207us total) trans 4294967448 rc 0/0 00000100:00100000:1.0:1789762893.417627:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067480064:12345-192.168.203.37@tcp:103:rm.0 Request processed in 75us (179us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762893.417645:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 65 00000100:00100000:0.0:1789762893.417649:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 94 00000100:00100000:2.0:1789762893.418370:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762893.418410:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067480576 00000100:00100000:3.0:1789762893.418416:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067480576 00000100:00100000:3.0:1789762893.418449:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 319 00000100:00100000:3.0:1789762893.418459:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:15178:x1876702067480576:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:3.0:1789762893.418484:0:11692:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1789762893.418497:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c762d800/0xdf5a7ac59f176acf lrc: 0/0,0 mode: --/CR res: [0x240000402:0x14:0x0].0x0 bits 0x5/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca1c expref: 40 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.418513:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769600/0xdf5a7ac59f17696a lrc: 0/0,0 mode: --/PR res: [0x240000402:0x14:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c951 expref: 39 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.418524:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762893.418528:0:11692:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762893.418570:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.418586:0:11692:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.418598:0:11692:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762893.418618:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fdfe2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074717568:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1789762893.418665:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762893.418708:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074717568 00000100:00100000:3.0:1789762893.418713:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d25693c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762893.418714:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074717568 00000100:00100000:2.0:1789762893.418747:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:2.0:1789762893.418758:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074717568:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1789762893.418770:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762893.418783:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176b1c lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f176b15 expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.418806:0:6424:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f1768fa lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c92e expref: 5 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.418815:0:6424:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176b1c lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f176b15 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.418833:0:6424:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f1768fa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c92e expref: 5 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.418850:0:6424:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f1768fa lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c92e expref: 5 pid: 6418 timeout: 466 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762893.418859:0:6424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2528a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074717696:192.168.203.37@tcp:104: 00000100:00100000:2.0:1789762893.418902:0:6424:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1789762893.419779:0:6424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2528a80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074717696:192.168.203.37@tcp:104: 00010000:00010000:0.0:1789762893.419811:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176b1c lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176b15 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.419822:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762893.419827:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.419832:0:6424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176b1c lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176b15 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.419842:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5860400, rc 0) 00000100:00100000:2.0:1789762893.419875:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.419900:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:11692:x1876702074717568:12345-0@lo:101:mdt00_004.0 Request processed in 1143us (1232us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762893.419909:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000100:00100000:3.0:1789762893.419973:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067480704 00000100:00100000:3.0:1789762893.419979:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067480704 00000100:00100000:0.0:1789762893.419985:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fdfe2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074717568:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1789762893.420001:0:8070:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f1768fa lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c92e expref: 6 pid: 6418 timeout: 466 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.420003:0:11692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.420027:0:11692:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762893.420050:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 85 00000100:00100000:3.0:1789762893.420061:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067480704:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762893.420071:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762893.420077:0:8070:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f1768fa lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c92e expref: 6 pid: 6418 timeout: 466 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.420099:0:8070:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f1768fa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c92e expref: 6 pid: 6418 timeout: 466 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.420115:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d1768400/0xdf5a7ac59f1768fa lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c92e expref: 6 pid: 6418 timeout: 466 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.420126:0:8070:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762893.420138:0:8070:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176b1c lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176b15 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.420163:0:8070:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176b1c lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f176b15 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.420172:0:8070:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176b1c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f176b15 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.420208:0:8070:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176b1c lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176b15 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762893.420220:0:8070:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fdfe1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:8070:1876702074717824:0@lo:105: 00000100:00100000:3.0:1789762893.420251:0:8070:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762893.420279:0:8070:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d2569180 going to sleep for 11 seconds 00000100:00100000:1.0:1789762893.420297:0:6402:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1789762893.420308:0:6402:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074717824 00000100:00100000:1.0:1789762893.420311:0:6402:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074717824 00000100:00100000:1.0:1789762893.420340:0:6402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 55 00000100:00100000:1.0:1789762893.420346:0:6402:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb868050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:8070:x1876702074717824:12345-0@lo:105: 00010000:00010000:1.0:1789762893.420354:0:6402:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.420362:0:6402:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.420370:0:6402:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.420375:0:6402:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.420380:0:6402:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.420386:0:6402:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.420406:0:6402:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c36c9af7c00) 00010000:00010000:0.0:1789762893.420424:0:11692:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.420436:0:11692:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762893.420456:0:11692:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1789762893.420463:0:6402:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb868050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:8070:x1876702074717824:12345-0@lo:105: Request processed in 113us (208us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762893.420472:0:6402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 55 00000100:00100000:3.0:1789762893.420482:0:8070:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fdfe1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:8070:1876702074717824:0@lo:105: 00010000:00010000:0.0:1789762893.420485:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.420496:0:11692:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.420505:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762893.420506:0:11692:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762893.420524:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074717952:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1789762893.420542:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067480704:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 482us (667us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762893.420552:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 85 00000100:00100000:0.0:1789762893.420570:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762893.420597:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5b1c300 going to sleep for 11 seconds 00000100:00100000:3.0:1789762893.420612:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074717952 00000100:00100000:3.0:1789762893.420618:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074717952 00000100:00100000:3.0:1789762893.420651:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 367 00000100:00100000:3.0:1789762893.420661:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:11692:x1876702074717952:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1789762893.420670:0:7227:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762893.420683:0:7227:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7e00/0xdf5a7ac59f176b2a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f176b23 expref: -99 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.420769:0:7227:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7e00/0xdf5a7ac59f176b2a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f176b23 expref: 8 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.420781:0:7227:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7e00/0xdf5a7ac59f176b2a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176b23 expref: 8 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.420789:0:7227:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72bab00) 00010000:00010000:3.0:1789762893.420792:0:7227:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762893.420795:0:7227:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36c9af7e00, rc 0) 00000100:00100000:3.0:1789762893.420833:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074717952:12345-0@lo:101:mdt00_004.0 Request processed in 174us (265us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762893.420841:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 367 00000100:00100000:0.0:1789762893.420867:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074717952:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1789762893.420879:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f176b2a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.420890:0:11692:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f176b2a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762893.420904:0:11692:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1789762893.420980:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074718080:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762893.421009:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762893.421030:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5b1cd80 going to sleep for 11 seconds 00000100:00100000:2.0:1789762893.421081:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074718080 00000100:00100000:2.0:1789762893.421086:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074718080 00000100:00100000:2.0:1789762893.421113:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 368 00000100:00100000:2.0:1789762893.421118:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074718080:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789762893.421190:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074718080:12345-0@lo:1000:mdt00_004.0 Request processed in 70us (180us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762893.421199:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 368 00000100:00100000:0.0:1789762893.421219:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074718080:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762893.421260:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074718208:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762893.421287:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762893.421307:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5b1cd80 going to sleep for 11 seconds 00000100:00100000:1.0:1789762893.421320:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074718208 00000100:00100000:1.0:1789762893.421325:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074718208 00000100:00100000:1.0:1789762893.421348:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 369 00000100:00100000:1.0:1789762893.421354:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074718208:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762893.421485:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074718208:12345-0@lo:1000:mdt00_004.0 Request processed in 130us (198us total) trans 0 rc 1/1 00000100:00100000:1.0:1789762893.421490:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 369 00000100:00100000:0.0:1789762893.421517:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074718208:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1789762893.421553:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.421562:0:11692:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.421571:0:11692:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762893.421583:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074718336:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1789762893.421609:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762893.421630:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f5b1c300 going to sleep for 11 seconds 00000100:00100000:2.0:1789762893.421654:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074718336 00000100:00100000:2.0:1789762893.421660:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074718336 00000100:00100000:2.0:1789762893.421690:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:2.0:1789762893.421697:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074718336:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1789762893.421704:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762893.421713:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860800/0xdf5a7ac59f176b38 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f176b31 expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.421726:0:6424:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860800/0xdf5a7ac59f176b38 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f176b31 expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.421734:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860800/0xdf5a7ac59f176b38 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176b31 expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.421746:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37d3bee100) 00010000:00010000:2.0:1789762893.421748:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762893.421750:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5860800, rc 0) 00000100:00100000:2.0:1789762893.421778:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:11692:x1876702074718336:12345-0@lo:101:mdt00_004.0 Request processed in 83us (170us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762893.421783:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00100000:0.0:1789762893.421808:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074718336:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1789762893.421820:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f176b38 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.421830:0:11692:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f176b38 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.421848:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176b3f lrc: 3/0,1 mode: --/EX res: [0x240000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.421858:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176b3f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.421866:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176b3f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762893.422436:0:11692:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].12, 424 off41216 00000040:00080000:0.0:1789762893.422481:0:11692:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].12, 504 off42608 00000004:00080000:0.0:1789762893.422488:0:11692:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37f5b1cd80 with version 25 00000004:00080000:2.0:1789762893.422559:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 25 rpc_version 25 00000020:00100000:2.0:1789762893.422578:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1789762893.422585:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37d2528380 x1876702074718592/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1744/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1789762893.422597:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2528380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074718592:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762893.422624:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762893.422645:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252ff00 going to sleep for 11 seconds 00000100:00100000:3.0:1789762893.422667:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074718592 00000100:00100000:3.0:1789762893.422673:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074718592 00000100:00100000:3.0:1789762893.422703:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 371 00000100:00100000:3.0:1789762893.422710:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9a34050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7224:x1876702074718592:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1789762893.423248:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9a34050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7224:x1876702074718592:12345-0@lo:1000:osp_up0-1.0 Request processed in 537us (620us total) trans 4294967426 rc 0/0 00000100:00100000:3.0:1789762893.423257:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 371 00000100:00100000:2.0:1789762893.423308:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2528380 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074718592:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1789762893.423337:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14:0x0]: returning size 16384 00010000:00010000:0.0:1789762893.423351:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176b3f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423360:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176b3f lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423373:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176b3f lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423380:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.423384:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176b3f lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423391:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176b3f lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423403:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:0.0:1789762893.423406:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.423410:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176b3f lrc: 2/0,0 mode: --/EX res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423418:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176b3f lrc: 0/0,0 mode: --/EX res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423432:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f176b38 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423441:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f176b38 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423449:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f176b38 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423471:0:11692:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1789762893.423479:0:11692:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1789762893.423484:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.423491:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762893.423493:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.423498:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f176b38 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423511:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.423512:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314e00 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:1.0:1789762893.423518:0:8069:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f176b38 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762893.423531:0:8069:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f176b38 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423540:0:11692:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1789762893.423545:0:11692:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1789762893.423553:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f176b2a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.423553:0:8069:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f176b38 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.423561:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762893.423569:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.423576:0:8069:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1789762893.423581:0:11692:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f176b2a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.423587:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7c00 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:2.0:1789762893.423594:0:6408:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762893.423602:0:6408:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.423605:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f176b38 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.423613:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca314e00/0xdf5a7ac59f176b31 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f176b38 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.423624:0:6408:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.423629:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762893.423630:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074718720:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762893.423632:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762893.423636:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762893.423642:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:15178:x1876702067480576:12345-192.168.203.37@tcp:36:rm.0 Request processed in 5179us (5266us total) trans 4294967449 rc 0/0 00010000:00010000:2.0:1789762893.423653:0:6408:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1789762893.423658:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 319 00000100:00100000:3.0:1789762893.423687:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789762893.423689:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.423697:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c36c9af7c00/0xdf5a7ac59f176b15 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f176b1c expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.423710:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762893.423713:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762893.423717:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762893.423722:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074718720 00000100:00100000:3.0:1789762893.423724:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789762893.423730:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074718720 00000100:00100000:3.0:1789762893.423736:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d2528700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074718848:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789762893.423765:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762893.423769:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 66 00000100:00100000:0.0:1789762893.423779:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c92c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3424:x1876702074718720:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762893.423789:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762893.423803:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860800/0xdf5a7ac59f176b38 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x14:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f176b31 expref: 10 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.423816:0:6406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37d3bee100) 00000100:00100000:1.0:1789762893.423825:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074718848 00010000:00010000:0.0:1789762893.423828:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762893.423831:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074718848 00000100:00100000:1.0:1789762893.423867:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 67 00000100:00100000:1.0:1789762893.423875:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3424:x1876702074718848:12345-0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762893.423880:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c92c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3424:x1876702074718720:12345-0@lo:103:ldlm_bl.0 Request processed in 100us (192us total) trans 0 rc 0/0 00010000:00010000:1.0:1789762893.423885:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1789762893.423896:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074718720:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762893.423897:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 66 00010000:00010000:1.0:1789762893.423903:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176b1c lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f176b15 expref: 10 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.423965:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762893.423970:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762893.424013:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074718848:12345-0@lo:103:ldlm_bl.0 Request processed in 137us (247us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762893.424021:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 67 00000100:00100000:3.0:1789762893.424028:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d2528700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074718848:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789762893.440032:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762893.700014:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762893.700051:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067480832 00000100:00100000:3.0:1789762893.700060:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067480832 00000100:00100000:3.0:1789762893.700109:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 320 00000100:00100000:3.0:1789762893.700117:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2528700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:15189:x1876702067480832:12345-192.168.203.37@tcp:101:mkdir.0 00010000:00010000:3.0:1789762893.700133:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762893.700161:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7800/0xdf5a7ac59f176b46 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca23 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.700235:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6600/0xdf5a7ac59f176b4d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.700247:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6600/0xdf5a7ac59f176b4d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.700257:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6600/0xdf5a7ac59f176b4d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762893.700286:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762893.700289:0:7744:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762893.700303:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6600/0xdf5a7ac59f176b4d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.700321:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7800/0xdf5a7ac59f176b46 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca23 expref: 7 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.700332:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6600/0xdf5a7ac59f176b4d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca23 expref: 6 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.700349:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762893.700351:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762893.700354:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36c9af6600, rc 0) 00000100:00100000:3.0:1789762893.700402:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2528700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15189:x1876702067480832:12345-192.168.203.37@tcp:101:mkdir.0 Request processed in 286us (395us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762893.700408:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 320 00000100:00100000:1.0:1789762893.701504:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762893.701541:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067481088 00000100:00100000:2.0:1789762893.701546:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067481088 00000100:00100000:2.0:1789762893.701579:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 321 00000100:00100000:2.0:1789762893.701588:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15189:x1876702067481088:12345-192.168.203.37@tcp:36:mkdir.0 00010000:00010000:2.0:1789762893.701616:0:13389:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762893.701630:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6600/0xdf5a7ac59f176b4d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca23 expref: 6 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.701641:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762893.701645:0:13389:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762893.701692:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860a00/0xdf5a7ac59f176b54 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.701700:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860a00/0xdf5a7ac59f176b54 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.701706:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860a00/0xdf5a7ac59f176b54 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.701721:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.701728:0:13389:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7e00/0xdf5a7ac59f176b2a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f176b23 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.701734:0:13389:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 4/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.701758:0:13389:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7e00/0xdf5a7ac59f176b2a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xdf5a7ac59f176b23 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762893.701770:0:13389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d252b100 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:13389:1876702074718976:0@lo:104: 00000100:00100000:2.0:1789762893.701804:0:13389:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762893.701842:0:13389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252ff00 going to sleep for 11 seconds 00000100:00100000:1.0:1789762893.701884:0:6449:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074718976 00000100:00100000:1.0:1789762893.701891:0:6449:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074718976 00000100:00100000:1.0:1789762893.701953:0:6449:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 56 00000100:00100000:1.0:1789762893.701963:0:6449:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:13389:x1876702074718976:12345-0@lo:104: 00010000:00010000:1.0:1789762893.701973:0:6449:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f176b2a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.702030:0:6449:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1789762893.702035:0:6449:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:13389:x1876702074718976:12345-0@lo:104: Request processed in 75us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762893.702040:0:6449:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 56 00000100:00100000:3.0:1789762893.702044:0:13389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d252b100 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:13389:1876702074718976:0@lo:104: 00010000:00010000:0.0:1789762893.702046:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762893.702056:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f176b2a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.702062:0:13389:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.702065:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315c00 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:0.0:1789762893.702074:0:6407:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xdf5a7ac59f176b2a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762893.702082:0:6407:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xdf5a7ac59f176b2a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.702105:0:6407:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f176b2a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.702109:0:6407:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1789762893.702130:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f176b2a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.702135:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37ca315c00/0xdf5a7ac59f176b23 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f176b2a expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762893.702140:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:0.0:1789762893.702165:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762893.702165:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074719104:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762893.702170:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762893.702173:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762893.702207:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762893.702242:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074719104 00000100:00100000:1.0:1789762893.702246:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074719104 00010000:00010000:1.0:1789762893.702264:0:6405:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7e00/0xdf5a7ac59f176b2a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xdf5a7ac59f176b23 expref: 8 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762893.702297:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 86 00000100:00100000:1.0:1789762893.702309:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3427:x1876702074719104:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762893.702319:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:0.0:1789762893.711863:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967426 is committed 00010000:00010000:1.0:1789762893.713486:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7e00/0xdf5a7ac59f176b2a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xdf5a7ac59f176b23 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.713500:0:6405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72bab00) 00010000:00010000:1.0:1789762893.713504:0:6405:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.713511:0:6405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.713516:0:6405:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.713538:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762893.713549:0:13389:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.713560:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762893.713589:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074719104:12345-0@lo:103:ldlm_bl.0 Request processed in 11284us (11382us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762893.713599:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 86 00000100:00100000:0.0:1789762893.713619:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074719104:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762893.714591:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860e00/0xdf5a7ac59f176b62 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714604:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860e00/0xdf5a7ac59f176b62 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714615:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860e00/0xdf5a7ac59f176b62 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762893.714637:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a3855671 reply state 0000000063befa0e transno 4294967427 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860e00/0xdf5a7ac59f176b62 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714646:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860e00/0xdf5a7ac59f176b62 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714651:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762893.714653:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762893.714655:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762893.714658:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 16384 00010000:00010000:2.0:1789762893.714664:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860a00/0xdf5a7ac59f176b54 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714670:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860a00/0xdf5a7ac59f176b54 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714675:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860a00/0xdf5a7ac59f176b54 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714679:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860a00/0xdf5a7ac59f176b54 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714683:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762893.714687:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860a00/0xdf5a7ac59f176b54 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714692:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860a00/0xdf5a7ac59f176b54 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714698:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762893.714700:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762893.714708:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860a00/0xdf5a7ac59f176b54 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714713:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860a00/0xdf5a7ac59f176b54 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762893.714721:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a3855671 reply state 0000000063befa0e transno 4294967427 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714726:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.714731:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72bab00) 00010000:00010000:2.0:1789762893.714733:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762893.714734:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:2.0:1789762893.714787:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15189:x1876702067481088:12345-192.168.203.37@tcp:36:mkdir.0 Request processed in 13201us (13285us total) trans 4294967427 rc 0/0 00000100:00100000:2.0:1789762893.714793:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 321 00000100:00100000:2.0:1789762893.720578:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.720608:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067481216 00000100:00100000:0.0:1789762893.720615:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067481216 00000100:00100000:0.0:1789762893.720655:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 322 00000100:00100000:0.0:1789762893.720663:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2528a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15190:x1876702067481216:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762893.720680:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762893.720690:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176b69 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca2a expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.720732:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176b70 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.720741:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176b70 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.720748:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176b70 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762893.720765:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762893.720767:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762893.720780:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176b70 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.720795:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176b69 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca2a expref: 9 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.720807:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176b70 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca2a expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.720813:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762893.720816:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762893.720818:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca314600, rc 0) 00000100:00100000:0.0:1789762893.720851:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2528a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15190:x1876702067481216:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 190us (277us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762893.720856:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 322 00000100:00100000:2.0:1789762893.721914:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.721985:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067481472 00000100:00100000:0.0:1789762893.721990:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067481472 00000100:00100000:0.0:1789762893.722017:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 323 00000100:00100000:0.0:1789762893.722023:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15190:x1876702067481472:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762893.722030:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762893.722032:0:6420:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1789762893.722042:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176b70 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca2a expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722051:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762893.722054:0:6420:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762893.722059:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176b77 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca31 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722138:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176b7e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722158:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176b7e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722165:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176b7e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722173:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176b85 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722179:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176b85 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722183:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176b85 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722230:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176b8c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722236:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176b8c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722241:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176b8c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762893.722246:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 16384 00010000:00010000:0.0:1789762893.722253:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176b8c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722258:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176b8c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722263:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176b8c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722267:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.722269:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176b8c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722273:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176b8c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722282:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:0.0:1789762893.722283:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.722285:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176b8c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722289:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176b8c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722300:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176b7e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722305:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176b7e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722309:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176b7e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722313:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176b7e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722316:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.722318:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176b7e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722322:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176b7e lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722327:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762893.722329:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.722331:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176b7e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722338:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176b7e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722346:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176b85 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722350:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176b85 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722354:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176b85 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722358:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.722360:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176b85 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722364:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176b85 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722368:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72bab00) 00010000:00010000:0.0:1789762893.722370:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.722372:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176b85 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722376:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176b85 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722393:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176b77 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca31 expref: 8 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722404:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762893.722406:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.722410:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176b77 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca31 expref: 8 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.722418:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca315800, rc 0) 00000100:00100000:0.0:1789762893.722446:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15190:x1876702067481472:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 423us (534us total) trans 0 rc 301/301 00000100:00100000:0.0:1789762893.722451:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 323 00000100:00100000:1.0:1789762893.723195:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762893.723230:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067481600 00000100:00100000:2.0:1789762893.723235:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067481600 00000100:00100000:2.0:1789762893.723266:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 324 00000100:00100000:2.0:1789762893.723273:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15190:x1876702067481600:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762893.723280:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762893.723288:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861400/0xdf5a7ac59f176b93 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca38 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.723316:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860c00/0xdf5a7ac59f176b9a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.723332:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860c00/0xdf5a7ac59f176b9a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.723346:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860c00/0xdf5a7ac59f176b9a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762893.723365:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762893.723368:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762893.723377:0:6419:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860c00/0xdf5a7ac59f176b9a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.723390:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861400/0xdf5a7ac59f176b93 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca38 expref: 9 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.723402:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860c00/0xdf5a7ac59f176b9a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca38 expref: 8 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.723407:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762893.723409:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762893.723411:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c5860c00, rc 0) 00000100:00100000:2.0:1789762893.723440:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15190:x1876702067481600:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 168us (247us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762893.723445:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 324 00000100:00100000:2.0:1789762893.724331:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.724363:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067481856 00000100:00100000:0.0:1789762893.724368:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067481856 00000100:00100000:0.0:1789762893.724397:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 325 00000100:00100000:0.0:1789762893.724404:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15190:x1876702067481856:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762893.724411:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762893.724419:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176ba1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca3f expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724444:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176ba8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724451:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176ba8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724457:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176ba8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724464:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176baf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724469:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176baf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724474:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176baf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762893.724513:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 16384 00010000:00010000:0.0:1789762893.724566:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176ba8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724571:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176ba8 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724575:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176ba8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724580:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176ba8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724584:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.724586:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176ba8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724590:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176ba8 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724597:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762893.724599:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.724601:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176ba8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724609:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176ba8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724617:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176baf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724621:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176baf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724626:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176baf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724629:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.724632:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176baf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724636:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176baf lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724640:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72bab00) 00010000:00010000:0.0:1789762893.724642:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.724644:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176baf lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724647:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176baf lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724664:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176ba1 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca3f expref: 10 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724673:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762893.724675:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762893.724677:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176ba1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca3f expref: 10 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.724683:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca314800, rc 0) 00000100:00100000:0.0:1789762893.724711:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:15190:x1876702067481856:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 308us (382us total) trans 4294967428 rc 301/301 00000100:00100000:0.0:1789762893.724716:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 325 00000100:00100000:3.0:1789762893.725656:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762893.725694:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067482112 00000100:00100000:2.0:1789762893.725700:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067482112 00000100:00100000:2.0:1789762893.725737:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 326 00000100:00100000:2.0:1789762893.725746:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:15190:x1876702067482112:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:2.0:1789762893.725787:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176bbd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.725795:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176bbd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.725800:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176bbd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762893.725937:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e8d6c5cb reply state 00000000373d759d transno 4294967429 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176bbd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.725950:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176bbd lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.725956:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762893.725959:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:2.0:1789762893.725962:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762893.725968:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 16384 00000100:00100000:2.0:1789762893.726024:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15190:x1876702067482112:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 278us (368us total) trans 4294967429 rc 0/0 00000100:00100000:2.0:1789762893.726034:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 326 00000100:00100000:2.0:1789762893.726662:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762893.726694:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067482368 00000100:00100000:3.0:1789762893.726699:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067482368 00000100:00100000:3.0:1789762893.726729:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 95 00000100:00100000:3.0:1789762893.726735:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15190:x1876702067482368:12345-192.168.203.37@tcp:35:lfs.0 00000100:00100000:3.0:1789762893.726848:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:15190:x1876702067482368:12345-192.168.203.37@tcp:35:lfs.0 Request processed in 111us (185us total) trans 4294967430 rc 0/0 00000100:00100000:3.0:1789762893.726855:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 95 00000100:00100000:3.0:1789762893.730624:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762893.730655:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067482496 00000100:00100000:3.0:1789762893.730660:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067482496 00000100:00100000:3.0:1789762893.730694:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 327 00000100:00100000:3.0:1789762893.730708:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:15191:x1876702067482496:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:3.0:1789762893.730721:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762893.730730:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7800/0xdf5a7ac59f176bc4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca46 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730767:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6c00/0xdf5a7ac59f176bcb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730777:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6c00/0xdf5a7ac59f176bcb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730784:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6c00/0xdf5a7ac59f176bcb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730795:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6400/0xdf5a7ac59f176bd2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730803:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6400/0xdf5a7ac59f176bd2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730811:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6400/0xdf5a7ac59f176bd2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730875:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7a00/0xdf5a7ac59f176bd9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730882:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7a00/0xdf5a7ac59f176bd9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730888:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7a00/0xdf5a7ac59f176bd9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762893.730935:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 16384 00000004:00200000:3.0:1789762893.730938:0:13389:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762893.730950:0:13389:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7a00/0xdf5a7ac59f176bd9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730958:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6c00/0xdf5a7ac59f176bcb lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730963:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6c00/0xdf5a7ac59f176bcb lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730967:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6c00/0xdf5a7ac59f176bcb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730972:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6c00/0xdf5a7ac59f176bcb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730976:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af6c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762893.730979:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6c00/0xdf5a7ac59f176bcb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730984:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6c00/0xdf5a7ac59f176bcb lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.730994:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762893.730996:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762893.730999:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6c00/0xdf5a7ac59f176bcb lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.731004:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6c00/0xdf5a7ac59f176bcb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.731021:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6400/0xdf5a7ac59f176bd2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.731026:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6400/0xdf5a7ac59f176bd2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.731030:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6400/0xdf5a7ac59f176bd2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.731034:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af6400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762893.731037:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6400/0xdf5a7ac59f176bd2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.731042:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6400/0xdf5a7ac59f176bd2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.731048:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72bab00) 00010000:00010000:3.0:1789762893.731050:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762893.731052:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6400/0xdf5a7ac59f176bd2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.731056:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6400/0xdf5a7ac59f176bd2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.731074:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7800/0xdf5a7ac59f176bc4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca46 expref: 14 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.731085:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7a00/0xdf5a7ac59f176bd9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca46 expref: 13 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.731099:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:3.0:1789762893.731100:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762893.731102:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36c9af7a00, rc 0) 00000100:00100000:3.0:1789762893.731137:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:15191:x1876702067482496:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 431us (515us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762893.731142:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 327 00000100:00100000:2.0:1789762893.732107:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.732132:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067482752 00000100:00100000:0.0:1789762893.732137:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067482752 00000100:00100000:0.0:1789762893.732184:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 328 00000100:00100000:0.0:1789762893.732193:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:15191:x1876702067482752:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:0.0:1789762893.732215:0:6418:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762893.732226:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7a00/0xdf5a7ac59f176bd9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca46 expref: 13 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.732237:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:0.0:1789762893.732239:0:6418:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762893.732250:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176be0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.732257:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176be0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.732263:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176be0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762893.732370:0:6418:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007b9d6335 reply state 00000000acba9710 transno 4294967431 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176be0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.732379:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176be0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.732384:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:0.0:1789762893.732386:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:0.0:1789762893.732388:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762893.732391:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 16384 00000100:00100000:0.0:1789762893.732431:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15191:x1876702067482752:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 240us (324us total) trans 4294967431 rc 0/0 00000100:00100000:0.0:1789762893.732436:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 328 00000100:00100000:3.0:1789762893.751989:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.752025:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067482880 00000100:00100000:0.0:1789762893.752035:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067482880 00000100:00100000:0.0:1789762893.752083:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 329 00000100:00100000:0.0:1789762893.752092:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15197:x1876702067482880:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:0.0:1789762893.752112:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762893.752128:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176be7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca4d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752196:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176bee lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752208:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176bee lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752217:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176bee lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752228:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176bf5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752233:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176bf5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752238:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176bf5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752315:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176bfc lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752320:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176bfc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752325:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176bfc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762893.752340:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 16384 00000004:00200000:0.0:1789762893.752342:0:6420:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762893.752356:0:6420:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176bfc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752365:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176bee lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752370:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176bee lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752375:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176bee lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752382:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176bee lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752387:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.752390:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176bee lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752395:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176bee lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752404:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762893.752406:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.752409:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176bee lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752413:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314400/0xdf5a7ac59f176bee lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752421:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176bf5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752425:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176bf5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752429:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176bf5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752433:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.752434:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176bf5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752441:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176bf5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752446:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72bab00) 00010000:00010000:0.0:1789762893.752447:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.752449:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176bf5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752452:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315200/0xdf5a7ac59f176bf5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752469:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314800/0xdf5a7ac59f176be7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca4d expref: 16 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752477:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176bfc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca4d expref: 15 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.752482:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:0.0:1789762893.752483:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789762893.752485:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca315a00, rc 0) 00000100:00100000:0.0:1789762893.752533:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:15197:x1876702067482880:12345-192.168.203.37@tcp:101:dd.500 Request processed in 443us (552us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762893.752539:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 329 00000100:00100000:3.0:1789762893.753718:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.753752:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067483136 00000100:00100000:0.0:1789762893.753757:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067483136 00000100:00100000:0.0:1789762893.753783:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 330 00000100:00100000:0.0:1789762893.753788:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:15197:x1876702067483136:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:0.0:1789762893.753798:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762893.753805:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176c03 lrc: 2/0,0 mode: --/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca54 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.753860:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176c0a lrc: 3/0,1 mode: --/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.753868:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176c0a lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.753874:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176c0a lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.753883:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176c11 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.753887:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176c11 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.753892:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176c11 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762893.754324:0:6419:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x8:0x0], index 0: 0 00010000:00010000:0.0:1789762893.754461:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176c18 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754470:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176c18 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754480:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176c18 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754514:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176c0a lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754520:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176c0a lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754524:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176c0a lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754528:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176c0a lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754531:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.754533:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176c0a lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754537:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176c0a lrc: 5/0,0 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754543:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:0.0:1789762893.754544:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.754547:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176c0a lrc: 3/0,0 mode: --/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754551:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314a00/0xdf5a7ac59f176c0a lrc: 0/0,0 mode: --/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762893.754558:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b878886a reply state 00000000c289b5ec transno 4294967432 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176c11 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754565:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176c11 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754570:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x4009f790 (ffff9c37e37a7600) 00010000:00010000:0.0:1789762893.754571:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762893.754573:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762893.754597:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314000/0xdf5a7ac59f176c03 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca54 expref: 18 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754605:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176c18 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca54 expref: 17 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.754610:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff9c37e37a7e00) 00010000:00010000:0.0:1789762893.754612:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762893.754614:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca315c00, rc 0) 00000100:00100000:0.0:1789762893.754641:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:15197:x1876702067483136:12345-192.168.203.37@tcp:101:dd.500 Request processed in 853us (926us total) trans 4294967432 rc 0/0 00000100:00100000:0.0:1789762893.754651:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 330 00000100:00100000:2.0:1789762893.758050:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762893.758097:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067483264 00000100:00100000:3.0:1789762893.758105:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067483264 00000100:00100000:3.0:1789762893.758158:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 366525082800 time 366525194411 state 0x0 ref 1 00000100:00100000:3.0:1789762893.758170:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 366625194411 time 366525194411 state 0x1 ref 1 00000100:00100000:3.0:1789762893.758192:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 366625194411 time 367005270300 state 0x0 ref 1 00000100:00100000:3.0:1789762893.758202:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2398:x1876702067483264:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762893.758229:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762893.758234:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6c6c6a0/0xdf5a7ac59f176c26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.758245:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=367s v=(0 0) 00010000:00010000:3.0:1789762893.758248:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762893.758253:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f6c6c6a0/0xdf5a7ac59f176c26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.758279:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6c6c6a0/0xdf5a7ac59f176c26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762893.778973:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967928 is committed 00002000:00010000:0.0:1789762893.778985:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:0.0:1789762893.778990:0:8059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb mtime from disk: 0 -> 1789762893 00002000:00010000:0.0:1789762893.778993:0:8059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb ctime from disk: 0 -> 1789762893 00002000:00010000:0.0:1789762893.778995:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:0.0:1789762893.779008:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f6c6c6a0/0xdf5a7ac59f176c26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762893.779012:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967929 is committed 00010000:00010000:0.0:1789762893.779021:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f6c6c6a0/0xdf5a7ac59f176c26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.779029:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f6c6c6a0/0xdf5a7ac59f176c26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.779039:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af6a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762893.779044:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6c6c6a0/0xdf5a7ac59f176c26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.779050:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f6c6c6a0/0xdf5a7ac59f176c26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.779066:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f6c6c6a0/0xdf5a7ac59f176c26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.779071:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6c6c6a0/0xdf5a7ac59f176c26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762893.779161:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067483264:12345-192.168.203.37@tcp:4:dd.500 Request processed in 20943us (21094us total) trans 4294967929 rc 0/0 00000100:00100000:0.0:1789762893.779170:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 724 00000100:00100000:0.0:1789762893.779180:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 366625194411 time 367005270300 state 0x2 ref 0 00000100:00100000:3.0:1789762893.791797:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762893.791873:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067483392 00000100:00100000:2.0:1789762893.791882:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067483392 00000100:00100000:2.0:1789762893.791937:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 366625194411 time 367005270300 state 0x0 ref 1 00000100:00100000:2.0:1789762893.791958:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 367105270300 time 367005270300 state 0x1 ref 1 00000100:00100000:2.0:1789762893.791980:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 367105270300 time 367039058439 state 0x0 ref 1 00000100:00100000:2.0:1789762893.791990:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067483392:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762893.792021:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 1048576, mtime: 0x6aad9d4d, atime: 0x0, ctime: 0x6aad9d4d, blocks: 0x803 00010000:00010000:2.0:1789762893.792028:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.792038:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=367s v=(0 0) 00010000:00010000:2.0:1789762893.792042:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762893.792046:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.792056:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762893.803110:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967930 is committed 00002000:00010000:0.0:1789762893.803191:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:0.0:1789762893.803195:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:0.0:1789762893.803205:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.803214:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.803225:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.803230:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762893.803233:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.803239:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.803252:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.803257:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762893.803313:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067483392:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11322us (11517us total) trans 4294967930 rc 0/0 00000100:00100000:0.0:1789762893.803321:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 725 00000100:00100000:0.0:1789762893.803329:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 367105270300 time 367039058439 state 0x2 ref 0 00000100:00100000:3.0:1789762893.805573:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762893.805631:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067483520 00000100:00100000:1.0:1789762893.805640:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067483520 00000100:00100000:1.0:1789762893.805675:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 367105270300 time 367039058439 state 0x0 ref 1 00000100:00100000:1.0:1789762893.805685:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 367139058439 time 367039058439 state 0x1 ref 1 00000100:00100000:1.0:1789762893.805707:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367139058439 time 367052785261 state 0x0 ref 1 00000100:00100000:1.0:1789762893.805717:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067483520:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762893.805747:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 2097152, mtime: 0x6aad9d4d, atime: 0x0, ctime: 0x6aad9d4d, blocks: 0x1007 00010000:00010000:1.0:1789762893.805755:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.805769:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=367s v=(0 0) 00010000:00010000:1.0:1789762893.805774:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762893.805779:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.805788:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762893.815830:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967931 is committed 00002000:00010000:0.0:1789762893.815838:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:0.0:1789762893.815844:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:0.0:1789762893.815854:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.815868:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.815875:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.815879:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762893.815883:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.815888:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.815902:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.815907:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762893.815975:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067483520:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10258us (10401us total) trans 4294967931 rc 0/0 00000100:00100000:0.0:1789762893.815983:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 726 00000100:00100000:0.0:1789762893.815992:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367139058439 time 367052785261 state 0x2 ref 0 00000100:00100000:3.0:1789762893.818123:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762893.818176:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067483648 00000100:00100000:1.0:1789762893.818183:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067483648 00000100:00100000:1.0:1789762893.818219:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367139058439 time 367052785261 state 0x0 ref 1 00000100:00100000:1.0:1789762893.818229:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367152785261 time 367052785261 state 0x1 ref 1 00000001:00080000:1.0:1789762893.832240:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967449 is committed 00000004:00080000:1.0:1789762893.832261:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 18 committed 00000004:00080000:1.0:1789762893.832270:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 18 committed 00010000:00010000:0.0:1789762893.832286:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1768bb lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832298:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1768bb lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832304:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1768bb lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832310:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.832314:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1768bb lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832320:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1768bb lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832338:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:0.0:1789762893.832341:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.832344:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1768bb lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832349:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933a00/0xdf5a7ac59f1768bb lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1789762893.832352:0:7221:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9c37d041d240 refcount 2 committed 0 result 0 batchid 11 00000020:00080000:1.0:1789762893.832358:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c36d1824600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9c37d041d000 00000020:00080000:1.0:1789762893.832361:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x2:0x0].11 00000020:00080000:1.0:1789762893.832364:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c36d1824100 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9c36cb5ed400 00010000:00010000:0.0:1789762893.832365:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17692b lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1789762893.832369:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x2:0x0].11 00010000:00010000:0.0:1789762893.832376:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17692b lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832381:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17692b lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832387:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.832390:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17692b lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832396:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17692b lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832404:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:0.0:1789762893.832406:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789762893.832409:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17692b lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832413:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768600/0xdf5a7ac59f17692b lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832425:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176940 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832430:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176940 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762893.832433:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074719232:0@lo:6:osp-syn-0-1.0 00010000:00010000:0.0:1789762893.832437:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176940 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832445:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.832448:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176940 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832453:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176940 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832461:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:0.0:1789762893.832463:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762893.832466:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176940 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762893.832471:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6089180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074719360:0@lo:6:osp-syn-1-1.0 00010000:00010000:0.0:1789762893.832473:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c5933e00/0xdf5a7ac59f176940 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762893.832487:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1789762893.832495:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17697f lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832501:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17697f lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832506:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17697f lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832511:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1768800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.832514:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17697f lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762893.832515:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074719232 00010000:00010000:0.0:1789762893.832521:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17697f lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762893.832522:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074719232 00000100:00100000:2.0:1789762893.832530:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1789762893.832539:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x73550ecb (ffff9c37d3bee300) 00010000:00010000:0.0:1789762893.832542:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.832544:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17697f lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832548:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1768800/0xdf5a7ac59f17697f lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832562:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f1769b7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832570:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f1769b7 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762893.832574:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074719360 00010000:00010000:0.0:1789762893.832575:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f1769b7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762893.832578:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074719360 00010000:00010000:0.0:1789762893.832584:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762893.832587:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f1769b7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762893.832590:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00010000:00010000:0.0:1789762893.832598:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f1769b7 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762893.832606:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074719232:12345-0@lo:6:osp-syn-0-1.0 00010000:00010000:0.0:1789762893.832607:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x2b0e463d (ffff9c37d3bee200) 00010000:00010000:0.0:1789762893.832614:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762893.832616:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00010000:00010000:0.0:1789762893.832618:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f1769b7 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832624:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d1769a00/0xdf5a7ac59f1769b7 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1789762893.832630:0:8054:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:13 count 1 00000100:00080000:1.0:1789762893.832633:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762893.832633:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ffcc8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074719360:12345-0@lo:6:osp-syn-1-1.0 00010000:00010000:0.0:1789762893.832635:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f176ae4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789762893.832639:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762893.832647:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f176ae4 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832652:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f176ae4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762893.832654:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xd:0x0] initial LVB size: 8388608, mtime: 0x6aad9d4c, atime: 0x0, ctime: 0x6aad9d4c, blocks: 0x4007 00010000:00010000:0.0:1789762893.832656:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314c00 already unused, calling callback (ffffffffc1f644f0) 00002000:00080000:2.0:1789762893.832659:0:9038:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:13 count 1 00010000:00010000:0.0:1789762893.832662:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f176ae4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.832663:0:8054:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f176c3b lrc: 3/0,1 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832681:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f176ae4 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.832686:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=367s v=(0 0) 00010000:00010000:0.0:1789762893.832689:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x2b0e463d (ffff9c37d3bee200) 00002000:00010000:2.0:1789762893.832690:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0xd:0x0] initial LVB size: 8388608, mtime: 0x6aad9d4c, atime: 0x0, ctime: 0x6aad9d4c, blocks: 0x1 00010000:00010000:3.0:1789762893.832691:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762893.832695:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.832701:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f176ae4 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.832703:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f176c3b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.832705:0:9038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005212985a/0xdf5a7ac59f176c42 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832715:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f176ae4 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.832735:0:8054:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f176c3b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.832738:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=367s v=(0 0) 00010000:00010000:0.0:1789762893.832740:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176b00 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.832751:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762893.832766:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176b00 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.832767:0:8054:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f176c3b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.832768:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005212985a/0xdf5a7ac59f176c42 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832779:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176b00 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832800:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762893.832807:0:9038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005212985a/0xdf5a7ac59f176c42 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832808:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176b00 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762893.832809:0:8054:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f176c3b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832833:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176b00 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.832842:0:9038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005212985a/0xdf5a7ac59f176c42 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.832843:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14:0x0].0x73550ecb (ffff9c37d3bee300) 00010000:00010000:3.0:1789762893.832847:0:8054:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762893.832848:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762893.832854:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176b00 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.832863:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762893.832866:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315600/0xdf5a7ac59f176b00 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x14:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.832866:0:9038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005212985a/0xdf5a7ac59f176c42 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.832872:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f176c3b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.832898:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7800 already unused, calling callback (ffffffffc1554350) 00000020:00080000:0.0:1789762893.832899:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 11 committed batchid 10 00010000:00010000:1.0:1789762893.832904:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f176c3b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762893.832905:0:9038:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762893.832945:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f176c3b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.832964:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f176c3b lrc: 1/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.832970:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f176c3b lrc: 0/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.832986:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762893.832989:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005212985a/0xdf5a7ac59f176c42 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.832994:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5861400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762893.832997:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005212985a/0xdf5a7ac59f176c42 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.833003:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005212985a/0xdf5a7ac59f176c42 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.833012:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005212985a/0xdf5a7ac59f176c42 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.833017:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005212985a/0xdf5a7ac59f176c42 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762893.833032:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762893.833035:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762893.833037:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:0.0:1789762893.833163:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 10 00000100:00100000:3.0:1789762893.833608:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074719232:12345-0@lo:6:osp-syn-0-1.0 Request processed in 1006us (1119us total) trans 4294967932 rc 0/0 00000100:00100000:3.0:1789762893.833615:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000004:00080000:0.0:1789762893.833639:0:3427:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37e3bdd500/2, rc 0, transno 4294967932 00000100:00100000:0.0:1789762893.833655:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074719232:0@lo:6:osp-syn-0-1.0 00000100:00100000:2.0:1789762893.833747:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ffcc8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074719360:12345-0@lo:6:osp-syn-1-1.0 Request processed in 1117us (1218us total) trans 4294967416 rc 0/0 00000100:00100000:2.0:1789762893.833757:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000004:00080000:3.0:1789762893.833767:0:3424:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37f6089180/2, rc 0, transno 4294967416 00000100:00100000:3.0:1789762893.833783:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6089180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074719360:0@lo:6:osp-syn-1-1.0 00010000:00010000:2.0:1789762893.833915:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762893.905808:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367152785261 time 367152866691 state 0x0 ref 1 00000100:00100000:0.0:1789762893.905831:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca559880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067483648:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762893.905891:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 3145728, mtime: 0x6aad9d4d, atime: 0x0, ctime: 0x6aad9d4d, blocks: 0x1807 00010000:00010000:0.0:1789762893.905897:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.905910:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=367s v=(0 0) 00010000:00010000:0.0:1789762893.905914:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762893.905954:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.905966:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762893.920461:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967933 is committed 00002000:00010000:0.0:1789762893.920479:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 3145728 -> 4194304 00000001:00080000:3.0:1789762893.920485:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967932 is committed 00002000:00010000:0.0:1789762893.920486:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:0.0:1789762893.920501:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.920511:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.920518:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.920523:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762893.920526:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.920545:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.920558:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762893.920563:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762893.920625:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca559880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067483648:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14800us (102500us total) trans 4294967933 rc 0/0 00000100:00100000:0.0:1789762893.920633:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 727 00000100:00100000:0.0:1789762893.920643:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367152785261 time 367152866691 state 0x2 ref 0 00000100:00100000:3.0:1789762893.923226:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762893.923275:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067483776 00000100:00100000:0.0:1789762893.923285:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067483776 00000100:00100000:0.0:1789762893.923325:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367152785261 time 367152866691 state 0x0 ref 1 00000100:00100000:0.0:1789762893.923335:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367252866691 time 367152866691 state 0x1 ref 1 00000100:00100000:3.0:1789762894.005883:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367252866691 time 367252948551 state 0x0 ref 1 00000100:00100000:3.0:1789762894.005906:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067483776:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762894.005970:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 4194304, mtime: 0x6aad9d4d, atime: 0x0, ctime: 0x6aad9d4d, blocks: 0x2007 00010000:00010000:3.0:1789762894.005977:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001615fb8e/0xdf5a7ac59f176c50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762894.005990:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=367s v=(0 0) 00010000:00010000:3.0:1789762894.005994:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762894.005998:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001615fb8e/0xdf5a7ac59f176c50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762894.006026:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001615fb8e/0xdf5a7ac59f176c50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762894.017953:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967934 is committed 00002000:00010000:0.0:1789762894.017959:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:0.0:1789762894.017965:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:0.0:1789762894.017976:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001615fb8e/0xdf5a7ac59f176c50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.017988:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001615fb8e/0xdf5a7ac59f176c50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.017994:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001615fb8e/0xdf5a7ac59f176c50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.017999:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af6400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762894.018002:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001615fb8e/0xdf5a7ac59f176c50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.018009:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001615fb8e/0xdf5a7ac59f176c50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.018024:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001615fb8e/0xdf5a7ac59f176c50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.018033:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001615fb8e/0xdf5a7ac59f176c50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762894.018108:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067483776:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12208us (94882us total) trans 4294967934 rc 0/0 00000100:00100000:0.0:1789762894.018117:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 728 00000100:00100000:0.0:1789762894.018127:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367252866691 time 367252948551 state 0x2 ref 0 00000100:00100000:3.0:1789762894.020540:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762894.020576:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067483904 00000100:00100000:0.0:1789762894.020583:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067483904 00000100:00100000:0.0:1789762894.020621:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367252866691 time 367252948551 state 0x0 ref 1 00000100:00100000:0.0:1789762894.020632:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367352948551 time 367252948551 state 0x1 ref 1 00000100:00100000:0.0:1789762894.105976:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367352948551 time 367353041466 state 0x0 ref 1 00000100:00100000:0.0:1789762894.106000:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca55a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067483904:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762894.106061:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 5242880, mtime: 0x6aad9d4d, atime: 0x0, ctime: 0x6aad9d4d, blocks: 0x2807 00010000:00010000:0.0:1789762894.106068:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.106081:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=367s v=(0 0) 00010000:00010000:0.0:1789762894.106095:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762894.106101:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.106112:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762894.117818:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967935 is committed 00002000:00010000:0.0:1789762894.117830:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:0.0:1789762894.117835:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:0.0:1789762894.117857:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.117867:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.117875:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.117880:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762894.117884:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.117890:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.117904:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.117909:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176c57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762894.117998:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca55a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067483904:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12004us (97457us total) trans 4294967935 rc 0/0 00000100:00100000:0.0:1789762894.118007:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 729 00000100:00100000:0.0:1789762894.118016:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367352948551 time 367353041466 state 0x2 ref 0 00000100:00100000:3.0:1789762894.120599:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762894.120640:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067484032 00000100:00100000:0.0:1789762894.120649:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067484032 00000100:00100000:0.0:1789762894.120693:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367352948551 time 367353041466 state 0x0 ref 1 00000100:00100000:0.0:1789762894.120707:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367453041466 time 367353041466 state 0x1 ref 1 00000100:00100000:2.0:1789762894.206075:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367453041466 time 367453135803 state 0x0 ref 1 00000100:00100000:2.0:1789762894.206108:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067484032:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762894.206187:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 6291456, mtime: 0x6aad9d4d, atime: 0x0, ctime: 0x6aad9d4d, blocks: 0x3007 00010000:00010000:2.0:1789762894.206195:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f176c5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762894.206219:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=367s v=(0 0) 00010000:00010000:2.0:1789762894.206223:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762894.206227:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f176c5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762894.206238:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f176c5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762894.218301:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967936 is committed 00002000:00010000:0.0:1789762894.218316:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:0.0:1789762894.218321:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 12295 -> 14343 00010000:00010000:0.0:1789762894.218331:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f176c5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.218341:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f176c5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.218347:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f176c5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.218352:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5861c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762894.218356:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f176c5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.218367:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f176c5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.218381:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f176c5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.218386:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004fb036c1/0xdf5a7ac59f176c5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762894.218447:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067484032:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12351us (97848us total) trans 4294967936 rc 0/0 00000100:00100000:0.0:1789762894.218455:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 730 00000100:00100000:0.0:1789762894.218465:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367453041466 time 367453135803 state 0x2 ref 0 00000100:00100000:2.0:1789762894.220859:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762894.220902:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067484160 00000100:00100000:0.0:1789762894.220910:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067484160 00000100:00100000:0.0:1789762894.220966:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367453041466 time 367453135803 state 0x0 ref 1 00000100:00100000:0.0:1789762894.220981:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367553135803 time 367453135803 state 0x1 ref 1 00000100:00100000:3.0:1789762894.280030:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762894.280060:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762894.280080:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762894.280091:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067484288 00000100:00100000:0.0:1789762894.280103:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067484288 00000100:00100000:2.0:1789762894.280108:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067484416 00000100:00100000:3.0:1789762894.280108:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762894.280117:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067484416 00000100:00100000:1.0:1789762894.280152:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067484672 00000100:00100000:3.0:1789762894.280154:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762894.280159:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067484672 00000100:00100000:0.0:1789762894.280190:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 369 00000100:00100000:2.0:1789762894.280191:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067484544 00000100:00100000:3.0:1789762894.280195:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762894.280199:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067484544 00000100:00100000:0.0:1789762894.280209:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1550050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067484288:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762894.280210:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 219 00000100:00100000:1.0:1789762894.280224:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2396:x1876702067484672:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762894.280225:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067484928 00000100:00100000:3.0:1789762894.280231:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067484928 00000100:00100000:2.0:1789762894.280233:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067484800 00000100:00100000:2.0:1789762894.280239:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067484800 00000100:00100000:3.0:1789762894.280277:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 538 00000100:00100000:0.0:1789762894.280285:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1550050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067484288:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 77us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762894.280286:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2396:x1876702067484672:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 61us (199us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762894.280294:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 539 00000100:00100000:3.0:1789762894.280296:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:2396:x1876702067484416:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762894.280307:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 219 00000100:00100000:0.0:1789762894.280308:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 369 00000100:00100000:2.0:1789762894.280314:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7fba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067484544:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:1.0:1789762894.280343:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762894.280345:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:2396:x1876702067484416:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 57us (280us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762894.280348:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762894.280354:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 538 00000100:00100000:2.0:1789762894.280359:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7fba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067484544:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 51us (251us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762894.280366:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 539 00000100:00100000:0.0:1789762894.280378:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 540 00000100:00100000:3.0:1789762894.280386:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 541 00000100:00100000:0.0:1789762894.280391:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ae2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067484928:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762894.280401:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9929880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067484800:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762894.280437:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ae2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067484928:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 50us (295us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762894.280446:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 540 00000100:00100000:3.0:1789762894.280448:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9929880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067484800:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 49us (255us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762894.280455:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 541 00000100:00100000:1.0:1789762894.306204:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367553135803 time 367553272476 state 0x0 ref 1 00000100:00100000:1.0:1789762894.306222:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067484160:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762894.306263:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 7340032, mtime: 0x6aad9d4d, atime: 0x0, ctime: 0x6aad9d4d, blocks: 0x3807 00010000:00010000:1.0:1789762894.306271:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762894.306283:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=367s v=(0 0) 00010000:00010000:1.0:1789762894.306287:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762894.306292:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762894.306303:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762894.318396:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967937 is committed 00002000:00010000:0.0:1789762894.318453:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:0.0:1789762894.318457:0:8058:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb mtime from disk: 1789762893 -> 1789762894 00002000:00010000:0.0:1789762894.318459:0:8058:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb ctime from disk: 1789762893 -> 1789762894 00002000:00010000:0.0:1789762894.318461:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:0.0:1789762894.318470:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.318478:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.318485:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.318490:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762894.318493:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.318503:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.318516:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.318521:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176c65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762894.318574:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067484160:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12355us (97715us total) trans 4294967937 rc 0/0 00000100:00100000:0.0:1789762894.318582:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 731 00000100:00100000:0.0:1789762894.318590:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367553135803 time 367553272476 state 0x2 ref 0 00000100:00100000:3.0:1789762894.320618:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762894.320664:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067485056 00000100:00100000:0.0:1789762894.320671:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067485056 00000100:00100000:0.0:1789762894.320704:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367553135803 time 367553272476 state 0x0 ref 1 00000100:00100000:0.0:1789762894.320714:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367653272476 time 367553272476 state 0x1 ref 1 00000100:00100000:2.0:1789762894.406296:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367653272476 time 367653359778 state 0x0 ref 1 00000100:00100000:2.0:1789762894.406337:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c992b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067485056:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762894.406388:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 8388608, mtime: 0x6aad9d4e, atime: 0x0, ctime: 0x6aad9d4e, blocks: 0x4007 00010000:00010000:2.0:1789762894.406395:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c6c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762894.406420:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=367s v=(0 0) 00010000:00010000:2.0:1789762894.406424:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762894.406428:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762894.406439:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762894.417575:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967938 is committed 00002000:00010000:0.0:1789762894.417626:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 8388608 -> 9437184 00002000:00010000:0.0:1789762894.417631:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 16391 -> 18439 00010000:00010000:0.0:1789762894.417640:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.417654:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.417660:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.417665:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762894.417669:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.417675:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.417687:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.417693:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176c6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762894.417744:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c992b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067485056:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11414us (97127us total) trans 4294967938 rc 0/0 00000100:00100000:0.0:1789762894.417753:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 732 00000100:00100000:0.0:1789762894.417762:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367653272476 time 367653359778 state 0x2 ref 0 00000100:00100000:3.0:1789762894.420616:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762894.420659:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067485184 00000100:00100000:0.0:1789762894.420669:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067485184 00000100:00100000:0.0:1789762894.420711:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367653272476 time 367653359778 state 0x0 ref 1 00000100:00100000:0.0:1789762894.420727:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367753359778 time 367653359778 state 0x1 ref 1 00010000:00010000:3.0:1789762894.464071:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762894.506386:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367753359778 time 367753450145 state 0x0 ref 1 00000100:00100000:1.0:1789762894.506411:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067485184:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762894.506461:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 9437184, mtime: 0x6aad9d4e, atime: 0x0, ctime: 0x6aad9d4e, blocks: 0x4807 00010000:00010000:1.0:1789762894.506469:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176c73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762894.506482:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=367s v=(0 0) 00010000:00010000:1.0:1789762894.506486:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762894.506490:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176c73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762894.506501:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176c73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762894.518699:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967939 is committed 00002000:00010000:0.0:1789762894.518707:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 9437184 -> 10485760 00002000:00010000:0.0:1789762894.518715:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 18439 -> 20487 00010000:00010000:0.0:1789762894.518729:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176c73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.518743:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176c73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.518752:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176c73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.518758:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762894.518761:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176c73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.518768:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176c73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.518782:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176c73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.518788:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176c73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762894.518859:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2759180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067485184:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12455us (98240us total) trans 4294967939 rc 0/0 00000100:00100000:0.0:1789762894.518868:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 733 00000100:00100000:0.0:1789762894.518877:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367753359778 time 367753450145 state 0x2 ref 0 00000100:00100000:3.0:1789762894.521469:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762894.521510:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067485312 00000100:00100000:0.0:1789762894.521519:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067485312 00000100:00100000:0.0:1789762894.521566:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367753359778 time 367753450145 state 0x0 ref 1 00000100:00100000:0.0:1789762894.521582:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367853450145 time 367753450145 state 0x1 ref 1 00000100:00100000:1.0:1789762894.606468:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367853450145 time 367853525416 state 0x0 ref 1 00000100:00100000:1.0:1789762894.606493:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2758700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067485312:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762894.606545:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 10485760, mtime: 0x6aad9d4e, atime: 0x0, ctime: 0x6aad9d4e, blocks: 0x5007 00010000:00010000:1.0:1789762894.606553:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176c7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762894.606566:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=367s v=(0 0) 00010000:00010000:1.0:1789762894.606570:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762894.606574:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176c7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762894.606590:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176c7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762894.617482:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967940 is committed 00002000:00010000:0.0:1789762894.617493:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:0.0:1789762894.617498:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:0.0:1789762894.617508:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176c7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.617517:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176c7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.617523:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176c7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.617528:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762dc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762894.617531:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176c7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.617539:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176c7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.617552:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176c7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.617556:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176c7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762894.617609:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2758700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067485312:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11125us (96140us total) trans 4294967940 rc 0/0 00000100:00100000:0.0:1789762894.617617:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 734 00000100:00100000:0.0:1789762894.617626:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367853450145 time 367853525416 state 0x2 ref 0 00000100:00100000:0.0:1789762894.619961:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762894.620012:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067485440 00000100:00100000:2.0:1789762894.620021:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067485440 00000100:00100000:2.0:1789762894.620061:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367853450145 time 367853525416 state 0x0 ref 1 00000100:00100000:2.0:1789762894.620076:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367953525416 time 367853525416 state 0x1 ref 1 00000100:00100000:3.0:1789762894.706557:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367953525416 time 367953622838 state 0x0 ref 1 00000100:00100000:3.0:1789762894.706581:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067485440:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762894.706631:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 11534336, mtime: 0x6aad9d4e, atime: 0x0, ctime: 0x6aad9d4e, blocks: 0x5807 00010000:00010000:3.0:1789762894.706638:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176c81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762894.706651:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=367s v=(0 0) 00010000:00010000:3.0:1789762894.706655:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762894.706660:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176c81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762894.706671:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176c81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762894.718244:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967941 is committed 00002000:00010000:0.0:1789762894.718248:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 11534336 -> 12582912 00002000:00010000:0.0:1789762894.718254:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 22535 -> 24583 00010000:00010000:0.0:1789762894.718264:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176c81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.718278:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176c81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.718289:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176c81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.718303:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762894.718309:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176c81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.718319:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176c81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.718336:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176c81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.718341:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176c81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762894.718398:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067485440:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11825us (98437us total) trans 4294967941 rc 0/0 00000100:00100000:0.0:1789762894.718408:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 735 00000100:00100000:0.0:1789762894.718416:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367953525416 time 367953622838 state 0x2 ref 0 00000100:00100000:3.0:1789762894.720688:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762894.720721:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067485568 00000100:00100000:0.0:1789762894.720731:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067485568 00000100:00100000:0.0:1789762894.720766:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 367953525416 time 367953622838 state 0x0 ref 1 00000100:00100000:0.0:1789762894.720777:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368053622838 time 367953622838 state 0x1 ref 1 00000100:00100000:3.0:1789762894.806631:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368053622838 time 368053697443 state 0x0 ref 1 00000100:00100000:3.0:1789762894.806655:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067485568:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762894.806704:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 12582912, mtime: 0x6aad9d4e, atime: 0x0, ctime: 0x6aad9d4e, blocks: 0x6007 00010000:00010000:3.0:1789762894.806711:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176c88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762894.806737:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=368s v=(0 0) 00010000:00010000:3.0:1789762894.806740:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762894.806745:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176c88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762894.806759:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176c88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762894.818656:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967942 is committed 00002000:00010000:0.0:1789762894.818722:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 12582912 -> 13631488 00002000:00010000:0.0:1789762894.818727:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 24583 -> 26631 00010000:00010000:0.0:1789762894.818739:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176c88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.818749:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176c88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.818756:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176c88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.818761:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af6800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762894.818765:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176c88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.818771:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176c88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.818786:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176c88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.818791:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176c88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762894.818866:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067485568:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12217us (98175us total) trans 4294967942 rc 0/0 00000100:00100000:0.0:1789762894.818874:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 736 00000100:00100000:0.0:1789762894.818884:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368053622838 time 368053697443 state 0x2 ref 0 00000100:00100000:0.0:1789762894.821439:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762894.821495:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067485696 00000100:00100000:1.0:1789762894.821503:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067485696 00000100:00100000:1.0:1789762894.821545:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368053622838 time 368053697443 state 0x0 ref 1 00000100:00100000:1.0:1789762894.821556:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368153697443 time 368053697443 state 0x1 ref 1 00000100:00100000:2.0:1789762894.906710:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368153697443 time 368153770783 state 0x0 ref 1 00000100:00100000:2.0:1789762894.906756:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067485696:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762894.906809:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 13631488, mtime: 0x6aad9d4e, atime: 0x0, ctime: 0x6aad9d4e, blocks: 0x6807 00010000:00010000:2.0:1789762894.906818:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176c8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762894.906834:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=368s v=(0 0) 00010000:00010000:2.0:1789762894.906839:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762894.906846:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176c8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762894.906860:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176c8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762894.918893:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967943 is committed 00002000:00010000:0.0:1789762894.918894:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 13631488 -> 14680064 00002000:00010000:0.0:1789762894.918901:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:0.0:1789762894.918912:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176c8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.918938:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176c8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.918946:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176c8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.918952:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762894.918956:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176c8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.918962:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176c8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.918976:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176c8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762894.918982:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176c8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762894.919043:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067485696:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12295us (97603us total) trans 4294967943 rc 0/0 00000100:00100000:0.0:1789762894.919050:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 737 00000100:00100000:0.0:1789762894.919061:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368153697443 time 368153770783 state 0x2 ref 0 00000100:00100000:2.0:1789762894.921404:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762894.921470:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067485824 00000100:00100000:0.0:1789762894.921478:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067485824 00000100:00100000:0.0:1789762894.921513:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368153697443 time 368153770783 state 0x0 ref 1 00000100:00100000:0.0:1789762894.921525:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368253770783 time 368153770783 state 0x1 ref 1 00000100:00100000:1.0:1789762895.006837:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368253770783 time 368253904545 state 0x0 ref 1 00000100:00100000:1.0:1789762895.006875:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067485824:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:2.0:1789762895.006913:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789762895.006941:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 14680064, mtime: 0x6aad9d4e, atime: 0x0, ctime: 0x6aad9d4e, blocks: 0x7007 00010000:00010000:1.0:1789762895.006961:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176c96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.006975:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=368s v=(0 0) 00010000:00010000:1.0:1789762895.006978:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762895.006983:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176c96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.006994:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176c96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762895.018870:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967944 is committed 00002000:00010000:0.0:1789762895.018871:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 14680064 -> 15728640 00002000:00010000:0.0:1789762895.018877:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:0.0:1789762895.018887:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176c96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.018897:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176c96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.018914:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176c96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.018932:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762895.018935:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176c96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.018942:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176c96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.018956:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176c96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.018961:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176c96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762895.019026:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067485824:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12173us (97622us total) trans 4294967944 rc 0/0 00000100:00100000:0.0:1789762895.019033:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 738 00000100:00100000:0.0:1789762895.019042:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368253770783 time 368253904545 state 0x2 ref 0 00000100:00100000:3.0:1789762895.021345:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762895.021388:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067485952 00000100:00100000:0.0:1789762895.021395:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067485952 00000100:00100000:0.0:1789762895.021429:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368253770783 time 368253904545 state 0x0 ref 1 00000100:00100000:0.0:1789762895.021439:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368353904545 time 368253904545 state 0x1 ref 1 00000100:00100000:0.0:1789762895.106938:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368353904545 time 368353981375 state 0x0 ref 1 00000100:00100000:0.0:1789762895.106963:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067485952:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762895.107008:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 15728640, mtime: 0x6aad9d4e, atime: 0x0, ctime: 0x6aad9d4e, blocks: 0x7807 00010000:00010000:0.0:1789762895.107015:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176c9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.107033:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=368s v=(0 0) 00010000:00010000:0.0:1789762895.107036:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762895.107041:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176c9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.107051:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176c9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762895.118962:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967945 is committed 00002000:00010000:0.0:1789762895.118963:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 15728640 -> 16777216 00002000:00010000:0.0:1789762895.118970:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 30727 -> 32775 00010000:00010000:0.0:1789762895.118980:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176c9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.118990:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176c9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.118997:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176c9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.119003:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762895.119006:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176c9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.119012:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176c9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.119030:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176c9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.119036:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176c9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762895.119100:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067485952:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12145us (97754us total) trans 4294967945 rc 0/0 00000100:00100000:0.0:1789762895.119108:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 739 00000100:00100000:0.0:1789762895.119118:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368353904545 time 368353981375 state 0x2 ref 0 00000100:00100000:3.0:1789762895.121579:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762895.121648:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067486080 00000100:00100000:0.0:1789762895.121656:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067486080 00000100:00100000:0.0:1789762895.121692:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368353904545 time 368353981375 state 0x0 ref 1 00000100:00100000:0.0:1789762895.121702:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368453981375 time 368353981375 state 0x1 ref 1 00000100:00100000:1.0:1789762895.207040:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368453981375 time 368454105088 state 0x0 ref 1 00000100:00100000:1.0:1789762895.207074:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1ab8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067486080:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762895.207125:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 16777216, mtime: 0x6aad9d4e, atime: 0x0, ctime: 0x6aad9d4e, blocks: 0x8007 00010000:00010000:1.0:1789762895.207147:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007fa9ef27/0xdf5a7ac59f176ca4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.207164:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=368s v=(0 0) 00010000:00010000:1.0:1789762895.207169:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762895.207188:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007fa9ef27/0xdf5a7ac59f176ca4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.207203:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007fa9ef27/0xdf5a7ac59f176ca4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762895.218840:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967946 is committed 00002000:00010000:0.0:1789762895.218847:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 16777216 -> 17825792 00002000:00010000:0.0:1789762895.218852:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:0.0:1789762895.218862:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007fa9ef27/0xdf5a7ac59f176ca4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.218872:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007fa9ef27/0xdf5a7ac59f176ca4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.218883:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007fa9ef27/0xdf5a7ac59f176ca4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.218889:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762ce00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762895.218903:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007fa9ef27/0xdf5a7ac59f176ca4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.218909:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007fa9ef27/0xdf5a7ac59f176ca4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.218938:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007fa9ef27/0xdf5a7ac59f176ca4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.218943:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007fa9ef27/0xdf5a7ac59f176ca4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762895.219005:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1ab8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067486080:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11948us (97426us total) trans 4294967946 rc 0/0 00000100:00100000:0.0:1789762895.219012:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 740 00000100:00100000:0.0:1789762895.219021:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368453981375 time 368454105088 state 0x2 ref 0 00000100:00100000:3.0:1789762895.221201:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762895.221232:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067486208 00000100:00100000:0.0:1789762895.221239:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067486208 00000100:00100000:0.0:1789762895.221273:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368453981375 time 368454105088 state 0x0 ref 1 00000100:00100000:0.0:1789762895.221299:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368554105088 time 368454105088 state 0x1 ref 1 00000100:00100000:1.0:1789762895.307120:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368554105088 time 368554186766 state 0x0 ref 1 00000100:00100000:1.0:1789762895.307159:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067486208:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762895.307220:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 17825792, mtime: 0x6aad9d4e, atime: 0x0, ctime: 0x6aad9d4e, blocks: 0x8807 00010000:00010000:1.0:1789762895.307227:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176cab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.307240:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=368s v=(0 0) 00010000:00010000:1.0:1789762895.307244:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762895.307260:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176cab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.307271:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176cab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762895.318732:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967947 is committed 00002000:00010000:0.0:1789762895.318739:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 17825792 -> 18874368 00002000:00010000:0.0:1789762895.318744:0:8059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb mtime from disk: 1789762894 -> 1789762895 00002000:00010000:0.0:1789762895.318746:0:8059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb ctime from disk: 1789762894 -> 1789762895 00002000:00010000:0.0:1789762895.318748:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 34823 -> 36871 00010000:00010000:0.0:1789762895.318758:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176cab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.318767:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176cab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.318774:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176cab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.318779:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762d400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762895.318783:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176cab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.318789:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176cab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.318802:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176cab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.318807:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176cab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762895.318861:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067486208:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11713us (97660us total) trans 4294967947 rc 0/0 00000100:00100000:0.0:1789762895.318869:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 741 00000100:00100000:0.0:1789762895.318878:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368554105088 time 368554186766 state 0x2 ref 0 00000100:00100000:2.0:1789762895.321131:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762895.321186:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067486336 00000100:00100000:0.0:1789762895.321194:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067486336 00000100:00100000:0.0:1789762895.321228:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368554105088 time 368554186766 state 0x0 ref 1 00000100:00100000:0.0:1789762895.321239:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368654186766 time 368554186766 state 0x1 ref 1 00000004:00080000:0.0:1789762895.360078:0:9022:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 368607157217ns, next at 369607154287ns, rc = 0 00000100:00100000:1.0:1789762895.360091:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074719488:0@lo:13:osp-pre-1-1.0 00000004:00080000:2.0:1789762895.360109:0:9018:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 368607189482ns, next at 369607186188ns, rc = 0 00000100:00100000:0.0:1789762895.360121:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074719616:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762895.360146:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bde680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074719744:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762895.360201:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762895.360211:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074719872:0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.360254:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762895.360258:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762895.360290:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074719488 00000100:00100000:3.0:1789762895.360300:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074719488 00000100:00100000:1.0:1789762895.360302:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074720000:0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.360304:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074720384:0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.360308:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074720128:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762895.360359:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762895.360373:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074719744 00000100:00100000:3.0:1789762895.360380:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074719744 00000100:00100000:1.0:1789762895.360381:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762895.360383:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074720896:0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.360392:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762895.360417:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074720512:0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.360431:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762895.360432:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074720128 00000100:00100000:3.0:1789762895.360438:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074720128 00000100:00100000:2.0:1789762895.360443:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074720640:0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.360467:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074721408:0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.360471:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:2.0:1789762895.360489:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762895.360495:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074719488:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762895.360503:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762895.360513:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762895.360519:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074721152:0@lo:400:kworker.0 00002000:00000024:3.0:1789762895.360532:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 96927744 pending 25559040 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789762895.360541:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074721024:0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.360554:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074719872 00000100:00100000:2.0:1789762895.360560:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762895.360561:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074719872 00000100:00100000:0.0:1789762895.360597:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:2.0:1789762895.360598:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1789762895.360605:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762895.360610:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1550850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074719872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.360612:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074720768:0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.360613:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074719488:12345-0@lo:13:osp-pre-1-1.0 Request processed in 107us (398us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762895.360628:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000100:00100000:1.0:1789762895.360642:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074721536:0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.360663:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 223 00000100:00100000:2.0:1789762895.360669:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762895.360676:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074719744:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762895.360690:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1550850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074719872:12345-0@lo:400:kworker.0 Request processed in 78us (430us total) trans 0 rc 0/0 00002000:00000024:3.0:1789762895.360690:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 96927744 pending 25559040 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789762895.360701:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762895.360702:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00100000:2.0:1789762895.360704:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074721280:0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.360768:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762895.360768:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074719744:12345-0@lo:13:osp-pre-1-0.0 Request processed in 93us (515us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762895.360774:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074719488:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762895.360777:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074719872:0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.360785:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 223 00000100:00100000:2.0:1789762895.360821:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bde680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074719744:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762895.360825:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00000100:00100000:3.0:1789762895.360838:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074720128:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762895.360848:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074720512 00000100:00100000:0.0:1789762895.360849:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074720384 00002000:00000024:3.0:1789762895.360850:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 525795328 pending 0 free 3859808256 avail 3822059520 00000100:00100000:0.0:1789762895.360857:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074720384 00000100:00100000:1.0:1789762895.360866:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074720512 00000100:00080000:2.0:1789762895.360877:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762895.360882:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762895.360903:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074720000 00000100:00100000:2.0:1789762895.360908:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074721408 00000100:00100000:0.0:1789762895.360909:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074720000 00000100:00100000:3.0:1789762895.360911:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074720128:12345-0@lo:13:osp-pre-0-1.0 Request processed in 74us (522us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762895.360940:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074721408 00000100:00100000:1.0:1789762895.360942:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:0.0:1789762895.360953:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074720896 00000100:00100000:3.0:1789762895.360960:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00100000:0.0:1789762895.360961:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074720896 00000100:00100000:1.0:1789762895.360968:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074720512:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.360994:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074720640 00000100:00100000:0.0:1789762895.360997:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074720640 00000100:00100000:2.0:1789762895.361012:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074721152 00000100:00100000:0.0:1789762895.361024:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074721024 00000100:00100000:2.0:1789762895.361028:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074721152 00000100:00100000:3.0:1789762895.361029:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074720768 00000100:00100000:0.0:1789762895.361030:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074721024 00000100:00100000:1.0:1789762895.361038:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074720512:12345-0@lo:400:kworker.0 Request processed in 84us (537us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762895.361038:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074720768 00000100:00100000:1.0:1789762895.361051:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00100000:2.0:1789762895.361064:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074721536 00000100:00100000:0.0:1789762895.361070:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 542 00000100:00100000:2.0:1789762895.361071:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074721536 00000100:00100000:0.0:1789762895.361093:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074720384:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.361093:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 543 00000100:00100000:1.0:1789762895.361096:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074721280 00000100:00100000:1.0:1789762895.361104:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074721280 00000100:00100000:3.0:1789762895.361109:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074720000:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.361111:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:2.0:1789762895.361127:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074721408:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.361155:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074720384:12345-0@lo:400:kworker.0 Request processed in 76us (796us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762895.361162:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 542 00000100:00100000:1.0:1789762895.361166:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:1.0:1789762895.361179:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074721152:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.361184:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 544 00000100:00100000:3.0:1789762895.361193:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074720000:12345-0@lo:400:kworker.0 Request processed in 88us (814us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762895.361198:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074720896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.361204:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074721408:12345-0@lo:400:kworker.0 Request processed in 79us (689us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762895.361207:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 543 00000100:00100000:2.0:1789762895.361229:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00100000:3.0:1789762895.361246:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 545 00000100:00100000:1.0:1789762895.361252:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074721152:12345-0@lo:400:kworker.0 Request processed in 72us (693us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762895.361260:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074720896:12345-0@lo:400:kworker.0 Request processed in 64us (829us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762895.361261:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:3.0:1789762895.361262:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074720640:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.361272:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 544 00000100:00100000:1.0:1789762895.361276:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000100:00100000:2.0:1789762895.361279:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074721536:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.361300:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 546 00000100:00100000:1.0:1789762895.361309:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 223 00000100:00100000:0.0:1789762895.361313:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074721024:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762895.361324:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074721280:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.361326:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074720640:12345-0@lo:400:kworker.0 Request processed in 67us (835us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762895.361339:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 545 00000100:00100000:2.0:1789762895.361353:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074721536:12345-0@lo:400:kworker.0 Request processed in 76us (655us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762895.361359:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074721024:12345-0@lo:400:kworker.0 Request processed in 49us (757us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762895.361363:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 547 00000100:00100000:2.0:1789762895.361372:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000100:00100000:0.0:1789762895.361383:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 546 00000100:00100000:1.0:1789762895.361387:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074721280:12345-0@lo:400:kworker.0 Request processed in 59us (613us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762895.361389:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074720768:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762895.361401:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 223 00000100:00080000:2.0:1789762895.361411:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762895.361417:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762895.361421:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762895.361426:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762895.361438:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074720768:12345-0@lo:400:kworker.0 Request processed in 63us (768us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762895.361446:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 547 00000100:00100000:2.0:1789762895.361453:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074719616 00000100:00100000:2.0:1789762895.361459:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074719616 00000100:00100000:0.0:1789762895.361461:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074720256:0@lo:400:kworker.0 00000100:00100000:1.0:1789762895.361487:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074720000:0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.361508:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:0.0:1789762895.361512:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762895.361514:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074720512:0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.361514:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074720768:0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.361526:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074719616:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762895.361543:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074721024:0@lo:400:kworker.0 00002000:00000024:2.0:1789762895.361543:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 525795328 pending 0 free 3859808256 avail 3822059520 00000100:00100000:3.0:1789762895.361548:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074721280:0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.361560:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074721408:0@lo:400:kworker.0 00000100:00100000:1.0:1789762895.361573:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074721536:0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.361579:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074720384:0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.361579:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074720640:0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.361602:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074720896:0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.361608:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074719616:12345-0@lo:13:osp-pre-0-0.0 Request processed in 85us (182us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762895.361611:0:3426:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294967932 gen 2 00000004:00080000:3.0:1789762895.361620:0:3426:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37e3bdd500, transno 4294967932 00000100:00100000:2.0:1789762895.361623:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00100000:1.0:1789762895.361626:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074720256 00000100:00100000:1.0:1789762895.361631:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074720256 00000100:00100000:3.0:1789762895.361639:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074721152:0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.361675:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074720128:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762895.361681:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 373 00000100:00100000:2.0:1789762895.361684:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f6088380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074719616:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762895.361696:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074720256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762895.361761:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074720256:12345-0@lo:400:kworker.0 Request processed in 71us (253us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762895.361768:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 373 00000100:00100000:2.0:1789762895.361770:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967426 gen 2 00000004:00080000:2.0:1789762895.361777:0:3427:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967426 committed transno 4294967426 00000020:00080000:2.0:1789762895.361783:0:3427:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9c37f5b1c780 refcount 2 committed 0 result 0 batchid 12 00000020:00080000:2.0:1789762895.361798:0:3427:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37ff763000 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9c37f5b1cd80 00000020:00080000:2.0:1789762895.361802:0:3427:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x2:0x0].12 00000020:00080000:2.0:1789762895.361805:0:3427:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37ff763600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9c36cb099c00 00000020:00080000:2.0:1789762895.361808:0:3427:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x2:0x0].12 00000100:00100000:2.0:1789762895.361842:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074720256:0@lo:400:kworker.0 00000020:00080000:3.0:1789762895.361871:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 12 committed batchid 10 00000020:00080000:3.0:1789762895.361979:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 10 00000100:00100000:1.0:1789762895.407204:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368654186766 time 368654270005 state 0x0 ref 1 00000100:00100000:1.0:1789762895.407232:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067486336:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762895.407293:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 18874368, mtime: 0x6aad9d4f, atime: 0x0, ctime: 0x6aad9d4f, blocks: 0x9007 00010000:00010000:1.0:1789762895.407300:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176cb2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.407314:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=368s v=(0 0) 00010000:00010000:1.0:1789762895.407317:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762895.407322:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176cb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.407333:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176cb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762895.417902:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967948 is committed 00002000:00010000:0.0:1789762895.417938:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:0.0:1789762895.417943:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:0.0:1789762895.417955:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176cb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.417965:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176cb2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.417972:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176cb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.417977:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762d200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762895.417981:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176cb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.417991:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176cb2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.418004:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176cb2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.418009:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176cb2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762895.418065:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067486336:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10842us (96933us total) trans 4294967948 rc 0/0 00000100:00100000:0.0:1789762895.418073:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 742 00000100:00100000:0.0:1789762895.418082:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368654186766 time 368654270005 state 0x2 ref 0 00000100:00100000:3.0:1789762895.420413:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762895.420446:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067486464 00000100:00100000:0.0:1789762895.420454:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067486464 00000100:00100000:0.0:1789762895.420492:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368654186766 time 368654270005 state 0x0 ref 1 00000100:00100000:0.0:1789762895.420511:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368754270005 time 368654270005 state 0x1 ref 1 00010000:00010000:3.0:1789762895.488060:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762895.507285:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368754270005 time 368754348211 state 0x0 ref 1 00000100:00100000:1.0:1789762895.507311:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067486464:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762895.507358:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 19922944, mtime: 0x6aad9d4f, atime: 0x0, ctime: 0x6aad9d4f, blocks: 0x9807 00010000:00010000:1.0:1789762895.507366:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008474e749/0xdf5a7ac59f176cb9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.507390:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=368s v=(0 0) 00010000:00010000:1.0:1789762895.507394:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762895.507398:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008474e749/0xdf5a7ac59f176cb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.507413:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008474e749/0xdf5a7ac59f176cb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762895.518571:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 19922944 -> 20971520 00000001:00080000:2.0:1789762895.518575:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967949 is committed 00002000:00010000:0.0:1789762895.518577:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:0.0:1789762895.518591:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008474e749/0xdf5a7ac59f176cb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.518605:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008474e749/0xdf5a7ac59f176cb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.518611:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008474e749/0xdf5a7ac59f176cb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.518616:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762d600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762895.518620:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008474e749/0xdf5a7ac59f176cb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.518626:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008474e749/0xdf5a7ac59f176cb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.518639:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008474e749/0xdf5a7ac59f176cb9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.518656:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008474e749/0xdf5a7ac59f176cb9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762895.518715:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067486464:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11413us (98301us total) trans 4294967949 rc 0/0 00000100:00100000:0.0:1789762895.518733:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 743 00000100:00100000:0.0:1789762895.518743:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368754270005 time 368754348211 state 0x2 ref 0 00000100:00100000:3.0:1789762895.521112:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762895.521156:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067486592 00000100:00100000:0.0:1789762895.521164:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067486592 00000100:00100000:0.0:1789762895.521202:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368754270005 time 368754348211 state 0x0 ref 1 00000100:00100000:0.0:1789762895.521213:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368854348211 time 368754348211 state 0x1 ref 1 00000100:00100000:2.0:1789762895.607375:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368854348211 time 368854440096 state 0x0 ref 1 00000100:00100000:2.0:1789762895.607414:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067486592:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762895.607465:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 20971520, mtime: 0x6aad9d4f, atime: 0x0, ctime: 0x6aad9d4f, blocks: 0xa007 00010000:00010000:2.0:1789762895.607473:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cc0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762895.607498:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=368s v=(0 0) 00010000:00010000:2.0:1789762895.607502:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762895.607506:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762895.607517:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762895.618490:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967950 is committed 00002000:00010000:0.0:1789762895.618491:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:0.0:1789762895.618497:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:0.0:1789762895.618506:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.618516:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cc0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.618523:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.618528:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762895.618532:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.618539:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cc0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.618553:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cc0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.618558:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cc0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762895.618619:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067486592:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11208us (97506us total) trans 4294967950 rc 0/0 00000100:00100000:0.0:1789762895.618632:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 744 00000100:00100000:0.0:1789762895.618644:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368854348211 time 368854440096 state 0x2 ref 0 00000100:00100000:3.0:1789762895.621427:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762895.621463:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067486720 00000100:00100000:0.0:1789762895.621481:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067486720 00000100:00100000:0.0:1789762895.621515:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368854348211 time 368854440096 state 0x0 ref 1 00000100:00100000:0.0:1789762895.621526:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368954440096 time 368854440096 state 0x1 ref 1 00000100:00100000:1.0:1789762895.707473:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368954440096 time 368954540223 state 0x0 ref 1 00000100:00100000:1.0:1789762895.707498:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067486720:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762895.707544:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 22020096, mtime: 0x6aad9d4f, atime: 0x0, ctime: 0x6aad9d4f, blocks: 0xa807 00010000:00010000:1.0:1789762895.707550:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176cc7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.707563:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=368s v=(0 0) 00010000:00010000:1.0:1789762895.707567:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762895.707572:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176cc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.707582:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176cc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762895.718862:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967951 is committed 00002000:00010000:0.0:1789762895.718873:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 22020096 -> 23068672 00002000:00010000:0.0:1789762895.718878:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:0.0:1789762895.718888:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176cc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.718897:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176cc7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.718903:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176cc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.718908:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762895.718912:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176cc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.718933:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176cc7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.718948:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176cc7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.718953:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176cc7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762895.719010:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067486720:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11520us (97584us total) trans 4294967951 rc 0/0 00000100:00100000:0.0:1789762895.719019:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 745 00000100:00100000:0.0:1789762895.719028:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368954440096 time 368954540223 state 0x2 ref 0 00000100:00100000:3.0:1789762895.721399:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762895.721446:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067486848 00000100:00100000:0.0:1789762895.721455:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067486848 00000100:00100000:0.0:1789762895.721509:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 368954440096 time 368954540223 state 0x0 ref 1 00000100:00100000:0.0:1789762895.721520:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369054540223 time 368954540223 state 0x1 ref 1 00000100:00100000:2.0:1789762895.807555:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369054540223 time 369054617596 state 0x0 ref 1 00000100:00100000:2.0:1789762895.807580:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067486848:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762895.807649:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 23068672, mtime: 0x6aad9d4f, atime: 0x0, ctime: 0x6aad9d4f, blocks: 0xb007 00010000:00010000:2.0:1789762895.807658:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762895.807671:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=369s v=(0 0) 00010000:00010000:2.0:1789762895.807675:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762895.807692:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762895.807706:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762895.819147:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967952 is committed 00002000:00010000:0.0:1789762895.819158:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:0.0:1789762895.819163:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:0.0:1789762895.819173:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.819188:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.819195:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.819201:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762895.819205:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.819211:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.819225:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.819230:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762895.819291:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067486848:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11718us (97891us total) trans 4294967952 rc 0/0 00000100:00100000:0.0:1789762895.819301:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 746 00000100:00100000:0.0:1789762895.819310:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369054540223 time 369054617596 state 0x2 ref 0 00000100:00100000:2.0:1789762895.821652:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762895.821696:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067486976 00000100:00100000:0.0:1789762895.821706:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067486976 00000100:00100000:0.0:1789762895.821751:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369054540223 time 369054617596 state 0x0 ref 1 00000100:00100000:0.0:1789762895.821766:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369154617596 time 369054617596 state 0x1 ref 1 00000100:00100000:1.0:1789762895.907648:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369154617596 time 369154702456 state 0x0 ref 1 00000100:00100000:1.0:1789762895.907682:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067486976:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762895.907736:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 24117248, mtime: 0x6aad9d4f, atime: 0x0, ctime: 0x6aad9d4f, blocks: 0xb807 00010000:00010000:1.0:1789762895.907743:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176cd5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.907764:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=369s v=(0 0) 00010000:00010000:1.0:1789762895.907768:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762895.907773:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176cd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762895.907785:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176cd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762895.920341:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967953 is committed 00002000:00010000:0.0:1789762895.920352:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 24117248 -> 25165824 00002000:00010000:0.0:1789762895.920357:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:0.0:1789762895.920369:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176cd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.920379:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176cd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.920385:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176cd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.920390:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762d800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762895.920394:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176cd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.920400:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176cd5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.920414:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176cd5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762895.920419:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176cd5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762895.920485:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067486976:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12817us (98830us total) trans 4294967953 rc 0/0 00000100:00100000:0.0:1789762895.920494:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 747 00000100:00100000:0.0:1789762895.920507:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369154617596 time 369154702456 state 0x2 ref 0 00000100:00100000:3.0:1789762895.923123:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762895.923192:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067487104 00000100:00100000:0.0:1789762895.923200:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067487104 00000100:00100000:0.0:1789762895.923245:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369154617596 time 369154702456 state 0x0 ref 1 00000100:00100000:0.0:1789762895.923258:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369254702456 time 369154702456 state 0x1 ref 1 00000100:00100000:1.0:1789762896.007737:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369254702456 time 369254796532 state 0x0 ref 1 00000100:00100000:1.0:1789762896.007766:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067487104:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762896.007827:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 25165824, mtime: 0x6aad9d4f, atime: 0x0, ctime: 0x6aad9d4f, blocks: 0xc007 00010000:00010000:1.0:1789762896.007834:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176cdc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.007848:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=369s v=(0 0) 00010000:00010000:1.0:1789762896.007851:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762896.007856:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176cdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.007868:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176cdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762896.019909:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967954 is committed 00002000:00010000:0.0:1789762896.019938:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:0.0:1789762896.019944:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:0.0:1789762896.019965:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176cdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.019975:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176cdc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.019983:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176cdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.019988:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762896.019991:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176cdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.019999:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176cdc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.020014:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176cdc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.020018:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176cdc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762896.020092:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067487104:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12335us (96968us total) trans 4294967954 rc 0/0 00000100:00100000:0.0:1789762896.020101:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 748 00000100:00100000:0.0:1789762896.020111:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369254702456 time 369254796532 state 0x2 ref 0 00000100:00100000:3.0:1789762896.022660:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762896.022693:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067487232 00000100:00100000:0.0:1789762896.022701:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067487232 00000100:00100000:0.0:1789762896.022738:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369254702456 time 369254796532 state 0x0 ref 1 00000100:00100000:0.0:1789762896.022749:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369354796532 time 369254796532 state 0x1 ref 1 00000100:00100000:1.0:1789762896.107803:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369354796532 time 369354869425 state 0x0 ref 1 00000100:00100000:1.0:1789762896.107828:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067487232:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762896.107875:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 26214400, mtime: 0x6aad9d4f, atime: 0x0, ctime: 0x6aad9d4f, blocks: 0xc807 00010000:00010000:1.0:1789762896.107882:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176ce3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.107895:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=369s v=(0 0) 00010000:00010000:1.0:1789762896.107899:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762896.107903:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176ce3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.107914:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176ce3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762896.119291:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967955 is committed 00002000:00010000:0.0:1789762896.119351:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 26214400 -> 27262976 00002000:00010000:0.0:1789762896.119356:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:0.0:1789762896.119377:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176ce3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.119391:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176ce3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.119397:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176ce3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.119402:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762896.119405:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176ce3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.119412:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176ce3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.119426:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176ce3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.119432:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176ce3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762896.119494:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067487232:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11671us (96833us total) trans 4294967955 rc 0/0 00000100:00100000:0.0:1789762896.119503:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 749 00000100:00100000:0.0:1789762896.119513:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369354796532 time 369354869425 state 0x2 ref 0 00000100:00100000:3.0:1789762896.122051:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762896.122094:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067487360 00000100:00100000:0.0:1789762896.122104:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067487360 00000100:00100000:0.0:1789762896.122164:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369354796532 time 369354869425 state 0x0 ref 1 00000100:00100000:0.0:1789762896.122176:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369454869425 time 369354869425 state 0x1 ref 1 00000100:00100000:1.0:1789762896.207876:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369454869425 time 369454940397 state 0x0 ref 1 00000100:00100000:1.0:1789762896.207901:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067487360:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762896.207969:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 27262976, mtime: 0x6aad9d4f, atime: 0x0, ctime: 0x6aad9d4f, blocks: 0xd007 00010000:00010000:1.0:1789762896.207976:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176cea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.207988:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=369s v=(0 0) 00010000:00010000:1.0:1789762896.207992:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762896.207997:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176cea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.208012:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176cea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762896.219541:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967956 is committed 00002000:00010000:0.0:1789762896.219606:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:0.0:1789762896.219611:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 53255 -> 55303 00010000:00010000:0.0:1789762896.219623:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176cea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.219636:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176cea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.219646:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176cea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.219655:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762d000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762896.219660:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176cea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.219667:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176cea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.219682:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176cea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.219691:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176cea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762896.219753:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067487360:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11860us (97700us total) trans 4294967956 rc 0/0 00000100:00100000:0.0:1789762896.219763:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 750 00000100:00100000:0.0:1789762896.219775:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369454869425 time 369454940397 state 0x2 ref 0 00000100:00100000:3.0:1789762896.221991:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762896.222022:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067487488 00000100:00100000:0.0:1789762896.222029:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067487488 00000100:00100000:0.0:1789762896.222062:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369454869425 time 369454940397 state 0x0 ref 1 00000100:00100000:0.0:1789762896.222072:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369554940397 time 369454940397 state 0x1 ref 1 00000100:00100000:2.0:1789762896.307967:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369554940397 time 369555032790 state 0x0 ref 1 00000100:00100000:2.0:1789762896.307987:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067487488:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762896.308031:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 28311552, mtime: 0x6aad9d4f, atime: 0x0, ctime: 0x6aad9d4f, blocks: 0xd807 00010000:00010000:2.0:1789762896.308046:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cf1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762896.308061:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=369s v=(0 0) 00010000:00010000:2.0:1789762896.308065:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762896.308069:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762896.308079:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762896.320155:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967957 is committed 00002000:00010000:0.0:1789762896.320156:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 28311552 -> 29360128 00002000:00010000:0.0:1789762896.320162:0:14170:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb mtime from disk: 1789762895 -> 1789762896 00002000:00010000:0.0:1789762896.320165:0:14170:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb ctime from disk: 1789762895 -> 1789762896 00002000:00010000:0.0:1789762896.320167:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 55303 -> 57351 00010000:00010000:0.0:1789762896.320176:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.320184:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cf1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.320191:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.320196:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762896.320199:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.320206:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.320221:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cf1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.320229:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176cf1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762896.320290:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067487488:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12308us (98299us total) trans 4294967957 rc 0/0 00000100:00100000:0.0:1789762896.320298:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 751 00000100:00100000:0.0:1789762896.320307:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369554940397 time 369555032790 state 0x2 ref 0 00000100:00100000:3.0:1789762896.322459:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762896.322492:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067487616 00000100:00100000:0.0:1789762896.322499:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067487616 00000100:00100000:0.0:1789762896.322532:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369554940397 time 369555032790 state 0x0 ref 1 00000100:00100000:0.0:1789762896.322550:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369655032790 time 369555032790 state 0x1 ref 1 00000100:00100000:1.0:1789762896.408072:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369655032790 time 369655137508 state 0x0 ref 1 00000100:00100000:1.0:1789762896.408097:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067487616:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762896.408156:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 29360128, mtime: 0x6aad9d50, atime: 0x0, ctime: 0x6aad9d50, blocks: 0xe007 00010000:00010000:1.0:1789762896.408163:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176cf8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.408177:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=369s v=(0 0) 00010000:00010000:1.0:1789762896.408180:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762896.408185:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.408196:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762896.419788:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967958 is committed 00002000:00010000:0.0:1789762896.419791:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 29360128 -> 30408704 00002000:00010000:0.0:1789762896.419798:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 57351 -> 59399 00010000:00010000:0.0:1789762896.419811:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.419821:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176cf8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.419829:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176cf8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.419834:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762896.419837:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176cf8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.419843:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176cf8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.419857:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176cf8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.419865:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176cf8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762896.419951:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067487616:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11862us (97493us total) trans 4294967958 rc 0/0 00000100:00100000:0.0:1789762896.419960:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 752 00000100:00100000:0.0:1789762896.419969:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369655032790 time 369655137508 state 0x2 ref 0 00000100:00100000:3.0:1789762896.422423:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762896.422465:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067487744 00000100:00100000:0.0:1789762896.422473:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067487744 00000100:00100000:0.0:1789762896.422507:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369655032790 time 369655137508 state 0x0 ref 1 00000100:00100000:0.0:1789762896.422518:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369755137508 time 369655137508 state 0x1 ref 1 00000100:00100000:2.0:1789762896.508194:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369755137508 time 369755259397 state 0x0 ref 1 00000100:00100000:2.0:1789762896.508219:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067487744:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762896.508285:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 30408704, mtime: 0x6aad9d50, atime: 0x0, ctime: 0x6aad9d50, blocks: 0xe807 00010000:00010000:2.0:1789762896.508292:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762896.508305:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=369s v=(0 0) 00010000:00010000:2.0:1789762896.508309:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762896.508314:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762896.508325:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.512373:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789762896.519354:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967959 is committed 00002000:00010000:0.0:1789762896.519412:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 30408704 -> 31457280 00002000:00010000:0.0:1789762896.519417:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 59399 -> 61447 00010000:00010000:0.0:1789762896.519439:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.519449:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.519456:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.519462:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762896.519465:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.519472:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.519485:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.519490:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176cff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762896.519557:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067487744:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11345us (97134us total) trans 4294967959 rc 0/0 00000100:00100000:0.0:1789762896.519565:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 753 00000100:00100000:0.0:1789762896.519574:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369755137508 time 369755259397 state 0x2 ref 0 00000100:00100000:2.0:1789762896.521879:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762896.521932:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067487872 00000100:00100000:0.0:1789762896.521940:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067487872 00000100:00100000:0.0:1789762896.521976:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369755137508 time 369755259397 state 0x0 ref 1 00000100:00100000:0.0:1789762896.521987:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369855259397 time 369755259397 state 0x1 ref 1 00000100:00100000:1.0:1789762896.608272:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369855259397 time 369855333773 state 0x0 ref 1 00000100:00100000:1.0:1789762896.608298:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067487872:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762896.608358:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 31457280, mtime: 0x6aad9d50, atime: 0x0, ctime: 0x6aad9d50, blocks: 0xf007 00010000:00010000:1.0:1789762896.608367:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.608385:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=369s v=(0 0) 00010000:00010000:1.0:1789762896.608389:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762896.608396:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.608410:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762896.619210:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967960 is committed 00002000:00010000:0.0:1789762896.619273:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 31457280 -> 32505856 00002000:00010000:0.0:1789762896.619278:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 61447 -> 63495 00010000:00010000:0.0:1789762896.619288:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.619298:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.619304:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.619309:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762da00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762896.619312:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.619320:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d06 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.619333:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d06 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.619339:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762896.619391:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067487872:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11102us (97512us total) trans 4294967960 rc 0/0 00000100:00100000:0.0:1789762896.619399:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 754 00000100:00100000:0.0:1789762896.619409:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369855259397 time 369855333773 state 0x2 ref 0 00000100:00100000:3.0:1789762896.621580:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762896.621621:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067488000 00000100:00100000:0.0:1789762896.621631:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067488000 00000100:00100000:0.0:1789762896.621667:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369855259397 time 369855333773 state 0x0 ref 1 00000100:00100000:0.0:1789762896.621678:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369955333773 time 369855333773 state 0x1 ref 1 00000100:00100000:1.0:1789762896.708339:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369955333773 time 369955403818 state 0x0 ref 1 00000100:00100000:1.0:1789762896.708362:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067488000:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762896.708411:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 32505856, mtime: 0x6aad9d50, atime: 0x0, ctime: 0x6aad9d50, blocks: 0xf807 00010000:00010000:1.0:1789762896.708418:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.708432:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=369s v=(0 0) 00010000:00010000:1.0:1789762896.708436:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762896.708441:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.708452:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762896.719784:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967961 is committed 00002000:00010000:0.0:1789762896.719853:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 32505856 -> 33554432 00002000:00010000:0.0:1789762896.719858:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 63495 -> 65543 00010000:00010000:0.0:1789762896.719867:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.719881:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.719892:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.719900:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762ca00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762896.719905:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.719935:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.719952:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d0d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.719958:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762896.720011:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067488000:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11657us (98431us total) trans 4294967961 rc 0/0 00000100:00100000:0.0:1789762896.720019:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 755 00000100:00100000:0.0:1789762896.720028:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369955333773 time 369955403818 state 0x2 ref 0 00000100:00100000:3.0:1789762896.722338:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762896.722381:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067488128 00000100:00100000:0.0:1789762896.722389:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067488128 00000100:00100000:0.0:1789762896.722425:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 369955333773 time 369955403818 state 0x0 ref 1 00000100:00100000:0.0:1789762896.722436:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370055403818 time 369955403818 state 0x1 ref 1 00000100:00100000:1.0:1789762896.808417:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370055403818 time 370055479077 state 0x0 ref 1 00000100:00100000:1.0:1789762896.808446:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee763800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067488128:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762896.808512:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 33554432, mtime: 0x6aad9d50, atime: 0x0, ctime: 0x6aad9d50, blocks: 0x10007 00010000:00010000:1.0:1789762896.808522:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.808540:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=370s v=(0 0) 00010000:00010000:1.0:1789762896.808545:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762896.808552:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.808571:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762896.819320:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967962 is committed 00002000:00010000:0.0:1789762896.819326:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:0.0:1789762896.819331:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 65543 -> 67591 00010000:00010000:0.0:1789762896.819341:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.819352:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.819359:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.819364:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762ca00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762896.819367:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.819373:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d14 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.819387:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d14 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.819392:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000496311b/0xdf5a7ac59f176d14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762896.819446:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee763800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067488128:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11011us (97109us total) trans 4294967962 rc 0/0 00000100:00100000:0.0:1789762896.819454:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 756 00000100:00100000:0.0:1789762896.819464:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370055403818 time 370055479077 state 0x2 ref 0 00000100:00100000:3.0:1789762896.821658:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762896.821694:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067488256 00000100:00100000:0.0:1789762896.821702:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067488256 00000100:00100000:0.0:1789762896.821735:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370055403818 time 370055479077 state 0x0 ref 1 00000100:00100000:0.0:1789762896.821746:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370155479077 time 370055479077 state 0x1 ref 1 00000100:00100000:1.0:1789762896.908497:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370155479077 time 370155562265 state 0x0 ref 1 00000100:00100000:1.0:1789762896.908521:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee761c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067488256:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762896.908569:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 34603008, mtime: 0x6aad9d50, atime: 0x0, ctime: 0x6aad9d50, blocks: 0x10807 00010000:00010000:1.0:1789762896.908576:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.908599:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=370s v=(0 0) 00010000:00010000:1.0:1789762896.908604:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762896.908609:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762896.908619:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762896.920710:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 34603008 -> 35651584 00000001:00080000:3.0:1789762896.920710:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967963 is committed 00002000:00010000:0.0:1789762896.920717:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 67591 -> 69639 00010000:00010000:0.0:1789762896.920728:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.920737:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.920744:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.920750:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762da00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762896.920754:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.920761:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.920775:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d1b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762896.920780:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a3570a5/0xdf5a7ac59f176d1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762896.920835:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee761c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067488256:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12323us (99176us total) trans 4294967963 rc 0/0 00000100:00100000:0.0:1789762896.920844:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 757 00000100:00100000:0.0:1789762896.920853:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370155479077 time 370155562265 state 0x2 ref 0 00000100:00100000:3.0:1789762896.923396:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762896.923430:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067488384 00000100:00100000:0.0:1789762896.923438:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067488384 00000100:00100000:0.0:1789762896.923475:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370155479077 time 370155562265 state 0x0 ref 1 00000100:00100000:0.0:1789762896.923486:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370255562265 time 370155562265 state 0x1 ref 1 00000100:00100000:2.0:1789762897.008623:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370255562265 time 370255687360 state 0x0 ref 1 00000100:00100000:2.0:1789762897.008662:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067488384:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:0.0:1789762897.008683:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762897.008713:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 35651584, mtime: 0x6aad9d50, atime: 0x0, ctime: 0x6aad9d50, blocks: 0x11007 00010000:00010000:2.0:1789762897.008720:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176d22 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762897.008733:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=370s v=(0 0) 00010000:00010000:2.0:1789762897.008737:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762897.008742:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176d22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762897.008753:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176d22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762897.019536:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967964 is committed 00002000:00010000:0.0:1789762897.019547:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 35651584 -> 36700160 00002000:00010000:0.0:1789762897.019552:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 69639 -> 71687 00010000:00010000:0.0:1789762897.019562:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176d22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.019576:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176d22 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.019583:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176d22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.019589:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762897.019593:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176d22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.019600:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176d22 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.019614:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176d22 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.019619:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176d22 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762897.019672:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067488384:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11032us (96275us total) trans 4294967964 rc 0/0 00000100:00100000:0.0:1789762897.019679:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 758 00000100:00100000:0.0:1789762897.019687:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370255562265 time 370255687360 state 0x2 ref 0 00000100:00100000:3.0:1789762897.022311:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762897.022350:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067488512 00000100:00100000:0.0:1789762897.022360:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067488512 00000100:00100000:0.0:1789762897.022405:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370255562265 time 370255687360 state 0x0 ref 1 00000100:00100000:0.0:1789762897.022421:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370355687360 time 370255687360 state 0x1 ref 1 00000100:00100000:1.0:1789762897.108707:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370355687360 time 370355773538 state 0x0 ref 1 00000100:00100000:1.0:1789762897.108727:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfeed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067488512:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762897.108770:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 36700160, mtime: 0x6aad9d50, atime: 0x0, ctime: 0x6aad9d50, blocks: 0x11807 00010000:00010000:1.0:1789762897.108777:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176d29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.108791:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=370s v=(0 0) 00010000:00010000:1.0:1789762897.108794:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762897.108799:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176d29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.108810:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176d29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762897.120238:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967965 is committed 00002000:00010000:0.0:1789762897.120247:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 36700160 -> 37748736 00002000:00010000:0.0:1789762897.120252:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 71687 -> 73735 00010000:00010000:0.0:1789762897.120264:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176d29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.120277:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176d29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.120288:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176d29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.120297:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762897.120302:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176d29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.120313:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176d29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.120332:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176d29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.120340:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176d29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762897.120403:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfeed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067488512:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11679us (98089us total) trans 4294967965 rc 0/0 00000100:00100000:0.0:1789762897.120414:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 759 00000100:00100000:0.0:1789762897.120426:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370355687360 time 370355773538 state 0x2 ref 0 00000100:00100000:3.0:1789762897.123050:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762897.123087:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067488640 00000100:00100000:0.0:1789762897.123095:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067488640 00000100:00100000:0.0:1789762897.123146:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370355687360 time 370355773538 state 0x0 ref 1 00000100:00100000:0.0:1789762897.123157:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370455773538 time 370355773538 state 0x1 ref 1 00000100:00100000:2.0:1789762897.208806:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370455773538 time 370455871803 state 0x0 ref 1 00000100:00100000:2.0:1789762897.208832:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067488640:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762897.208883:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 37748736, mtime: 0x6aad9d50, atime: 0x0, ctime: 0x6aad9d50, blocks: 0x12007 00010000:00010000:2.0:1789762897.208891:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176d30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762897.208904:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=370s v=(0 0) 00010000:00010000:2.0:1789762897.208908:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762897.208913:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176d30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762897.208944:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176d30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762897.220228:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967966 is committed 00002000:00010000:0.0:1789762897.220244:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:0.0:1789762897.220250:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 73735 -> 75783 00010000:00010000:0.0:1789762897.220262:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176d30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.220276:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176d30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.220287:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176d30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.220295:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762897.220301:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176d30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.220313:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176d30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.220332:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176d30 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.220340:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176d30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762897.220407:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067488640:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11582us (97355us total) trans 4294967966 rc 0/0 00000100:00100000:0.0:1789762897.220419:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 760 00000100:00100000:0.0:1789762897.220433:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370455773538 time 370455871803 state 0x2 ref 0 00000100:00100000:2.0:1789762897.223130:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762897.223166:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067488768 00000100:00100000:0.0:1789762897.223175:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067488768 00000100:00100000:0.0:1789762897.223213:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370455773538 time 370455871803 state 0x0 ref 1 00000100:00100000:0.0:1789762897.223224:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370555871803 time 370455871803 state 0x1 ref 1 00000100:00100000:1.0:1789762897.308891:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370555871803 time 370555951582 state 0x0 ref 1 00000100:00100000:1.0:1789762897.308914:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067488768:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762897.308986:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 38797312, mtime: 0x6aad9d50, atime: 0x0, ctime: 0x6aad9d50, blocks: 0x12807 00010000:00010000:1.0:1789762897.308993:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176d37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.309006:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=370s v=(0 0) 00010000:00010000:1.0:1789762897.309010:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762897.309014:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176d37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.309026:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176d37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762897.320489:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967967 is committed 00002000:00010000:0.0:1789762897.320512:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 38797312 -> 39845888 00002000:00010000:0.0:1789762897.320519:0:13300:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb mtime from disk: 1789762896 -> 1789762897 00002000:00010000:0.0:1789762897.320523:0:13300:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb ctime from disk: 1789762896 -> 1789762897 00002000:00010000:0.0:1789762897.320526:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 75783 -> 77831 00010000:00010000:0.0:1789762897.320539:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176d37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.320553:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176d37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.320563:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176d37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.320572:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762d000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762897.320577:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176d37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.320588:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176d37 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.320606:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176d37 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762897.320614:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176d37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762897.320688:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067488768:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11778us (97568us total) trans 4294967967 rc 0/0 00000100:00100000:0.0:1789762897.320698:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 761 00000100:00100000:0.0:1789762897.320708:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370555871803 time 370555951582 state 0x2 ref 0 00000100:00100000:3.0:1789762897.323016:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762897.323047:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067488896 00000100:00100000:0.0:1789762897.323056:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067488896 00000100:00100000:0.0:1789762897.323093:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370555871803 time 370555951582 state 0x0 ref 1 00000100:00100000:0.0:1789762897.323104:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370655951582 time 370555951582 state 0x1 ref 1 00000100:00100000:1.0:1789762897.408065:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789762897.408086:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f608b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074721664:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762897.408217:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762897.408261:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074721664 00000100:00100000:0.0:1789762897.408269:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074721664 00000100:00100000:0.0:1789762897.408329:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 374 00000100:00100000:0.0:1789762897.408341:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6088380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074721664:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762897.408384:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762897.408447:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6088380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074721664:12345-0@lo:41:osp-pre-1-0.0 Request processed in 108us (229us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762897.408458:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 374 00000100:00100000:3.0:1789762897.408477:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f608b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074721664:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762897.408960:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370655951582 time 370656030857 state 0x0 ref 1 00000100:00100000:1.0:1789762897.408975:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067488896:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762897.409012:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 39845888, mtime: 0x6aad9d51, atime: 0x0, ctime: 0x6aad9d51, blocks: 0x13007 00010000:00010000:1.0:1789762897.409020:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176d3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.409033:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=370s v=(0 0) 00010000:00010000:1.0:1789762897.409036:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762897.409041:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176d3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.409053:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176d3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762897.422912:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967968 is committed 00002000:00010000:2.0:1789762897.422940:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 39845888 -> 40894464 00002000:00010000:2.0:1789762897.422945:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 77831 -> 79879 00010000:00010000:2.0:1789762897.422958:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176d3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762897.422969:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176d3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762897.422987:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176d3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762897.422993:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762897.422997:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176d3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762897.423004:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176d3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762897.423021:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176d3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762897.423027:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176d3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762897.423113:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067488896:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14137us (100093us total) trans 4294967968 rc 0/0 00000100:00100000:2.0:1789762897.423137:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 762 00000100:00100000:2.0:1789762897.423148:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370655951582 time 370656030857 state 0x2 ref 0 00000100:00100000:2.0:1789762897.425518:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762897.425560:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067489024 00000100:00100000:1.0:1789762897.425570:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067489024 00000100:00100000:1.0:1789762897.425613:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370655951582 time 370656030857 state 0x0 ref 1 00000100:00100000:1.0:1789762897.425625:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370756030857 time 370656030857 state 0x1 ref 1 00000001:00080000:0.0:1789762897.428607:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967414 is committed 00000001:00080000:0.0:1789762897.428646:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967416 is committed 00000001:00080000:0.0:1789762897.428662:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967415 is committed 00000100:00100000:1.0:1789762897.509035:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370756030857 time 370756097768 state 0x0 ref 1 00000100:00100000:1.0:1789762897.509087:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067489024:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762897.509160:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 40894464, mtime: 0x6aad9d51, atime: 0x0, ctime: 0x6aad9d51, blocks: 0x13807 00010000:00010000:1.0:1789762897.509168:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.509182:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=370s v=(0 0) 00010000:00010000:1.0:1789762897.509187:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762897.509194:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.509210:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762897.521360:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967969 is committed 00002000:00010000:1.0:1789762897.521381:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 40894464 -> 41943040 00002000:00010000:1.0:1789762897.521386:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 79879 -> 81927 00010000:00010000:1.0:1789762897.521398:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.521408:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.521415:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.521420:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762897.521423:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.521430:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d45 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.521445:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d45 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.521450:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762897.521517:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067489024:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12442us (95998us total) trans 4294967969 rc 0/0 00000100:00100000:1.0:1789762897.521526:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 763 00000100:00100000:1.0:1789762897.521536:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370756030857 time 370756097768 state 0x2 ref 0 00000100:00100000:3.0:1789762897.524026:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762897.524068:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067489152 00000100:00100000:0.0:1789762897.524076:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067489152 00000100:00100000:0.0:1789762897.524114:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370756030857 time 370756097768 state 0x0 ref 1 00000100:00100000:0.0:1789762897.524140:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370856097768 time 370756097768 state 0x1 ref 1 00010000:00010000:2.0:1789762897.535988:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762897.609144:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370856097768 time 370856209673 state 0x0 ref 1 00000100:00100000:1.0:1789762897.609168:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067489152:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762897.609215:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 41943040, mtime: 0x6aad9d51, atime: 0x0, ctime: 0x6aad9d51, blocks: 0x14007 00010000:00010000:1.0:1789762897.609222:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.609235:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=370s v=(0 0) 00010000:00010000:1.0:1789762897.609239:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762897.609245:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.609260:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762897.620084:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967970 is committed 00002000:00010000:1.0:1789762897.620109:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 41943040 -> 42991616 00002000:00010000:1.0:1789762897.620114:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 81927 -> 83975 00010000:00010000:1.0:1789762897.620137:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.620146:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.620157:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.620163:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762897.620167:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.620173:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.620186:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.620192:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762897.620247:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067489152:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11088us (96221us total) trans 4294967970 rc 0/0 00000100:00100000:1.0:1789762897.620255:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 764 00000100:00100000:1.0:1789762897.620264:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370856097768 time 370856209673 state 0x2 ref 0 00000100:00100000:3.0:1789762897.622446:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762897.622488:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067489280 00000100:00100000:2.0:1789762897.622502:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067489280 00000100:00100000:2.0:1789762897.622539:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370856097768 time 370856209673 state 0x0 ref 1 00000100:00100000:2.0:1789762897.622550:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370956209673 time 370856209673 state 0x1 ref 1 00000100:00100000:1.0:1789762897.709227:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370956209673 time 370956295222 state 0x0 ref 1 00000100:00100000:1.0:1789762897.709249:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067489280:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762897.709292:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 42991616, mtime: 0x6aad9d51, atime: 0x0, ctime: 0x6aad9d51, blocks: 0x14807 00010000:00010000:1.0:1789762897.709299:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.709312:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=370s v=(0 0) 00010000:00010000:1.0:1789762897.709316:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762897.709321:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.709331:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762897.720278:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967971 is committed 00002000:00010000:1.0:1789762897.720299:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 42991616 -> 44040192 00002000:00010000:1.0:1789762897.720304:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 83975 -> 86023 00010000:00010000:1.0:1789762897.720313:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.720321:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.720328:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.720333:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762897.720336:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.720347:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.720361:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.720366:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762897.720424:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfedc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067489280:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11182us (97987us total) trans 4294967971 rc 0/0 00000100:00100000:1.0:1789762897.720432:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 765 00000100:00100000:1.0:1789762897.720440:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370956209673 time 370956295222 state 0x2 ref 0 00000100:00100000:3.0:1789762897.722569:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762897.722637:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067489408 00000100:00100000:1.0:1789762897.722645:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067489408 00000100:00100000:1.0:1789762897.722682:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 370956209673 time 370956295222 state 0x0 ref 1 00000100:00100000:1.0:1789762897.722692:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371056295222 time 370956295222 state 0x1 ref 1 00000100:00100000:1.0:1789762897.809297:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371056295222 time 371056362845 state 0x0 ref 1 00000100:00100000:1.0:1789762897.809322:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067489408:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762897.809371:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 44040192, mtime: 0x6aad9d51, atime: 0x0, ctime: 0x6aad9d51, blocks: 0x15007 00010000:00010000:1.0:1789762897.809379:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d5a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.809393:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=371s v=(0 0) 00010000:00010000:1.0:1789762897.809396:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762897.809401:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.809414:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762897.821345:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967972 is committed 00002000:00010000:1.0:1789762897.821363:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 44040192 -> 45088768 00002000:00010000:1.0:1789762897.821369:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 86023 -> 88071 00010000:00010000:1.0:1789762897.821378:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.821387:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.821394:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.821399:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762897.821403:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.821409:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.821424:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.821429:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762897.821488:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067489408:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12174us (98918us total) trans 4294967972 rc 0/0 00000100:00100000:1.0:1789762897.821495:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 766 00000100:00100000:1.0:1789762897.821504:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371056295222 time 371056362845 state 0x2 ref 0 00000100:00100000:3.0:1789762897.823629:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762897.823674:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067489536 00000100:00100000:1.0:1789762897.823681:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067489536 00000100:00100000:1.0:1789762897.823716:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371056295222 time 371056362845 state 0x0 ref 1 00000100:00100000:1.0:1789762897.823727:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371156362845 time 371056362845 state 0x1 ref 1 00000100:00100000:1.0:1789762897.909364:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371156362845 time 371156429124 state 0x0 ref 1 00000100:00100000:1.0:1789762897.909390:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067489536:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762897.909439:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 45088768, mtime: 0x6aad9d51, atime: 0x0, ctime: 0x6aad9d51, blocks: 0x15807 00010000:00010000:1.0:1789762897.909447:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.909465:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=371s v=(0 0) 00010000:00010000:1.0:1789762897.909469:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762897.909473:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.909484:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762897.920858:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967973 is committed 00002000:00010000:1.0:1789762897.920910:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 45088768 -> 46137344 00002000:00010000:1.0:1789762897.920915:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 88071 -> 90119 00010000:00010000:1.0:1789762897.920942:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.920956:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.920966:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.920973:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762897.920978:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.920988:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.921013:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762897.921021:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007185cdb1/0xdf5a7ac59f176d61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762897.921096:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067489536:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11713us (97465us total) trans 4294967973 rc 0/0 00000100:00100000:1.0:1789762897.921107:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 767 00000100:00100000:1.0:1789762897.921135:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371156362845 time 371156429124 state 0x2 ref 0 00000100:00100000:3.0:1789762897.923410:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762897.923446:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067489664 00000100:00100000:1.0:1789762897.923453:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067489664 00000100:00100000:1.0:1789762897.923490:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371156362845 time 371156429124 state 0x0 ref 1 00000100:00100000:1.0:1789762897.923500:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371256429124 time 371156429124 state 0x1 ref 1 00000100:00100000:0.0:1789762898.009456:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371256429124 time 371256521145 state 0x0 ref 1 00000100:00100000:0.0:1789762898.009480:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fcfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067489664:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762898.009527:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 46137344, mtime: 0x6aad9d51, atime: 0x0, ctime: 0x6aad9d51, blocks: 0x16007 00010000:00010000:0.0:1789762898.009535:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176d68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.009549:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=371s v=(0 0) 00010000:00010000:0.0:1789762898.009553:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762898.009558:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176d68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.009569:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176d68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762898.021279:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967974 is committed 00002000:00010000:1.0:1789762898.021293:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 46137344 -> 47185920 00002000:00010000:1.0:1789762898.021298:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 90119 -> 92169 00010000:00010000:1.0:1789762898.021308:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176d68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.021318:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176d68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.021329:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176d68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.021334:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762898.021337:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176d68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.021343:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176d68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.021357:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176d68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.021362:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f176d68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.021420:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fcfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067489664:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11947us (98007us total) trans 4294967974 rc 0/0 00000100:00100000:1.0:1789762898.021429:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 768 00000100:00100000:1.0:1789762898.021438:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371256429124 time 371256521145 state 0x2 ref 0 00000100:00100000:3.0:1789762898.023665:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762898.023741:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067489792 00000100:00100000:1.0:1789762898.023749:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067489792 00000100:00100000:1.0:1789762898.023786:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371256429124 time 371256521145 state 0x0 ref 1 00000100:00100000:1.0:1789762898.023800:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371356521145 time 371256521145 state 0x1 ref 1 00000100:00100000:2.0:1789762898.109544:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371356521145 time 371356609627 state 0x0 ref 1 00000100:00100000:2.0:1789762898.109565:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067489792:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762898.109613:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 47185920, mtime: 0x6aad9d51, atime: 0x0, ctime: 0x6aad9d51, blocks: 0x16809 00010000:00010000:2.0:1789762898.109621:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176d6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.109635:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=371s v=(0 0) 00010000:00010000:2.0:1789762898.109639:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762898.109643:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176d6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.109655:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176d6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762898.120896:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967975 is committed 00002000:00010000:1.0:1789762898.120911:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 47185920 -> 48234496 00002000:00010000:1.0:1789762898.120939:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 92169 -> 94217 00010000:00010000:1.0:1789762898.120951:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176d6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.120961:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176d6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.120968:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176d6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.120983:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5861400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762898.120986:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176d6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.120996:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176d6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.121010:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176d6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.121016:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176d6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.121075:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067489792:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11516us (97411us total) trans 4294967975 rc 0/0 00000100:00100000:1.0:1789762898.121084:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 769 00000100:00100000:1.0:1789762898.121093:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371356521145 time 371356609627 state 0x2 ref 0 00000100:00100000:3.0:1789762898.123694:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762898.123743:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067489920 00000100:00100000:1.0:1789762898.123754:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067489920 00000100:00100000:1.0:1789762898.123794:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371356521145 time 371356609627 state 0x0 ref 1 00000100:00100000:1.0:1789762898.123805:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371456609627 time 371356609627 state 0x1 ref 1 00000100:00100000:0.0:1789762898.209629:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371456609627 time 371456694140 state 0x0 ref 1 00000100:00100000:0.0:1789762898.209659:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067489920:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762898.209712:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 48234496, mtime: 0x6aad9d51, atime: 0x0, ctime: 0x6aad9d51, blocks: 0x17009 00010000:00010000:0.0:1789762898.209720:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176d76 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.209733:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=371s v=(0 0) 00010000:00010000:0.0:1789762898.209736:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762898.209741:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176d76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.209756:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176d76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762898.220640:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967976 is committed 00002000:00010000:1.0:1789762898.220691:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 48234496 -> 49283072 00002000:00010000:1.0:1789762898.220696:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 94217 -> 96265 00010000:00010000:1.0:1789762898.220705:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176d76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.220714:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176d76 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.220721:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176d76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.220726:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762898.220729:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176d76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.220737:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176d76 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.220751:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176d76 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.220756:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176d76 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.220815:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067489920:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11167us (97120us total) trans 4294967976 rc 0/0 00000100:00100000:1.0:1789762898.220823:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 770 00000100:00100000:1.0:1789762898.220833:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371456609627 time 371456694140 state 0x2 ref 0 00000100:00100000:3.0:1789762898.223292:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762898.223331:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067490048 00000100:00100000:1.0:1789762898.223339:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067490048 00000100:00100000:1.0:1789762898.223376:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371456609627 time 371456694140 state 0x0 ref 1 00000100:00100000:1.0:1789762898.223387:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371556694140 time 371456694140 state 0x1 ref 1 00000100:00100000:0.0:1789762898.309717:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371556694140 time 371556785219 state 0x0 ref 1 00000100:00100000:0.0:1789762898.309735:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067490048:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762898.309774:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 49283072, mtime: 0x6aad9d51, atime: 0x0, ctime: 0x6aad9d51, blocks: 0x17809 00010000:00010000:0.0:1789762898.309781:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176d7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.309794:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=371s v=(0 0) 00010000:00010000:0.0:1789762898.309797:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762898.309802:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176d7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.309812:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176d7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762898.320761:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967977 is committed 00002000:00010000:1.0:1789762898.320779:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 49283072 -> 50331648 00002000:00010000:1.0:1789762898.320784:0:13301:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb mtime from disk: 1789762897 -> 1789762898 00002000:00010000:1.0:1789762898.320787:0:13301:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb ctime from disk: 1789762897 -> 1789762898 00002000:00010000:1.0:1789762898.320789:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 96265 -> 98313 00010000:00010000:1.0:1789762898.320798:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176d7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.320806:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176d7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.320813:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176d7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.320818:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762898.320821:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176d7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.320828:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176d7d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.320841:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176d7d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.320846:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f176d7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.320902:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067490048:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11172us (97610us total) trans 4294967977 rc 0/0 00000100:00100000:1.0:1789762898.320910:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 771 00000100:00100000:1.0:1789762898.320948:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371556694140 time 371556785219 state 0x2 ref 0 00000100:00100000:3.0:1789762898.323077:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762898.323120:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067490176 00000100:00100000:1.0:1789762898.323130:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067490176 00000100:00100000:1.0:1789762898.323166:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371556694140 time 371556785219 state 0x0 ref 1 00000100:00100000:1.0:1789762898.323177:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371656785219 time 371556785219 state 0x1 ref 1 00000100:00100000:0.0:1789762898.409815:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371656785219 time 371656883004 state 0x0 ref 1 00000100:00100000:0.0:1789762898.409837:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067490176:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762898.409880:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 50331648, mtime: 0x6aad9d52, atime: 0x0, ctime: 0x6aad9d52, blocks: 0x18009 00010000:00010000:0.0:1789762898.409887:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176d84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.409900:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=371s v=(0 0) 00010000:00010000:0.0:1789762898.409904:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762898.409909:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176d84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.409941:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176d84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762898.420787:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967978 is committed 00002000:00010000:1.0:1789762898.420809:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 50331648 -> 51380224 00002000:00010000:1.0:1789762898.420815:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 98313 -> 100361 00010000:00010000:1.0:1789762898.420824:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176d84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.420833:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176d84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.420841:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176d84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.420849:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762898.420854:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176d84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.420863:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176d84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.420882:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176d84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.420897:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176d84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.420991:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067490176:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11160us (97913us total) trans 4294967978 rc 0/0 00000100:00100000:1.0:1789762898.421000:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 772 00000100:00100000:1.0:1789762898.421009:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371656785219 time 371656883004 state 0x2 ref 0 00000100:00100000:3.0:1789762898.423127:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762898.423167:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067490304 00000100:00100000:1.0:1789762898.423176:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067490304 00000100:00100000:1.0:1789762898.423211:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371656785219 time 371656883004 state 0x0 ref 1 00000100:00100000:1.0:1789762898.423222:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371756883004 time 371656883004 state 0x1 ref 1 00000100:00100000:3.0:1789762898.432056:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074721792:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762898.432094:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762898.432143:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074721792 00000100:00100000:2.0:1789762898.432150:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074721792 00000100:00100000:2.0:1789762898.432192:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 375 00000100:00100000:2.0:1789762898.432200:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074721792:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762898.432236:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762898.432276:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074721792:12345-0@lo:41:osp-pre-0-1.0 Request processed in 77us (181us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762898.432282:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 375 00000100:00100000:3.0:1789762898.432320:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074721792:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789762898.509913:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371756883004 time 371756979530 state 0x0 ref 1 00000100:00100000:0.0:1789762898.509960:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067490304:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762898.510005:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 51380224, mtime: 0x6aad9d52, atime: 0x0, ctime: 0x6aad9d52, blocks: 0x18809 00010000:00010000:0.0:1789762898.510012:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f176d8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.510027:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=371s v=(0 0) 00010000:00010000:0.0:1789762898.510032:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762898.510039:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f176d8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.510054:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f176d8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762898.521123:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967979 is committed 00002000:00010000:1.0:1789762898.521186:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 51380224 -> 52428800 00002000:00010000:1.0:1789762898.521191:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 100361 -> 102409 00010000:00010000:1.0:1789762898.521200:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f176d8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.521209:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f176d8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.521215:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f176d8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.521225:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762898.521228:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f176d8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.521234:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f176d8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.521248:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f176d8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.521253:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f176d8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.521312:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067490304:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11358us (98199us total) trans 4294967979 rc 0/0 00000100:00100000:1.0:1789762898.521320:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 773 00000100:00100000:1.0:1789762898.521330:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371756883004 time 371756979530 state 0x2 ref 0 00000100:00100000:3.0:1789762898.523639:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762898.523676:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067490432 00000100:00100000:1.0:1789762898.523683:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067490432 00000100:00100000:1.0:1789762898.523719:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371756883004 time 371756979530 state 0x0 ref 1 00000100:00100000:1.0:1789762898.523729:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371856979530 time 371756979530 state 0x1 ref 1 00010000:00010000:3.0:1789762898.560041:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762898.610089:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371856979530 time 371857138280 state 0x0 ref 1 00000100:00100000:0.0:1789762898.610157:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067490432:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762898.610234:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 52428800, mtime: 0x6aad9d52, atime: 0x0, ctime: 0x6aad9d52, blocks: 0x19009 00010000:00010000:0.0:1789762898.610246:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.610265:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=371s v=(0 0) 00010000:00010000:0.0:1789762898.610270:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762898.610278:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.610294:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762898.629019:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967980 is committed 00002000:00010000:1.0:1789762898.629144:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 52428800 -> 53477376 00002000:00010000:1.0:1789762898.629153:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 102409 -> 104457 00010000:00010000:1.0:1789762898.629167:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.629181:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.629191:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.629198:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762898.629203:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.629213:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.629233:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.629240:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.629331:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067490432:12345-192.168.203.37@tcp:4:dd.500 Request processed in 19214us (105688us total) trans 4294967980 rc 0/0 00000100:00100000:1.0:1789762898.629344:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 774 00000100:00100000:1.0:1789762898.629357:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371856979530 time 371857138280 state 0x2 ref 0 00000100:00100000:1.0:1789762898.632911:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762898.633015:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067490560 00000100:00100000:0.0:1789762898.633028:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067490560 00000100:00100000:0.0:1789762898.633133:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371856979530 time 371857138280 state 0x0 ref 1 00000100:00100000:0.0:1789762898.633158:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371957138280 time 371857138280 state 0x1 ref 1 00000100:00100000:0.0:1789762898.710158:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371957138280 time 371957224045 state 0x0 ref 1 00000100:00100000:0.0:1789762898.710181:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067490560:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762898.710228:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 53477376, mtime: 0x6aad9d52, atime: 0x0, ctime: 0x6aad9d52, blocks: 0x19809 00010000:00010000:0.0:1789762898.710235:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.710249:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=371s v=(0 0) 00010000:00010000:0.0:1789762898.710253:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762898.710257:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.710268:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762898.722225:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967981 is committed 00002000:00010000:1.0:1789762898.722301:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 53477376 -> 54525952 00002000:00010000:1.0:1789762898.722305:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 104457 -> 106505 00010000:00010000:1.0:1789762898.722329:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.722339:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.722346:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.722351:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762898.722354:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.722361:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.722375:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.722379:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f176d99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.722448:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f608b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067490560:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12273us (89537us total) trans 4294967981 rc 0/0 00000100:00100000:1.0:1789762898.722458:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 775 00000100:00100000:1.0:1789762898.722467:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371957138280 time 371957224045 state 0x2 ref 0 00000100:00100000:3.0:1789762898.724895:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762898.724966:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067490688 00000100:00100000:1.0:1789762898.724974:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067490688 00000100:00100000:1.0:1789762898.725014:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 371957138280 time 371957224045 state 0x0 ref 1 00000100:00100000:1.0:1789762898.725036:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372057224045 time 371957224045 state 0x1 ref 1 00000100:00100000:2.0:1789762898.810252:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372057224045 time 372057318234 state 0x0 ref 1 00000100:00100000:2.0:1789762898.810275:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067490688:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762898.810326:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 54525952, mtime: 0x6aad9d52, atime: 0x0, ctime: 0x6aad9d52, blocks: 0x1a009 00010000:00010000:2.0:1789762898.810333:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176da0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.810346:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=372s v=(0 0) 00010000:00010000:2.0:1789762898.810349:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762898.810354:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176da0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.810366:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176da0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762898.821496:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967982 is committed 00002000:00010000:1.0:1789762898.821548:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 54525952 -> 55574528 00002000:00010000:1.0:1789762898.821553:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 106505 -> 108553 00010000:00010000:1.0:1789762898.821562:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176da0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.821571:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176da0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.821577:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176da0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.821582:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762898.821585:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176da0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.821592:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176da0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.821607:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176da0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.821612:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176da0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.821672:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067490688:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11404us (96777us total) trans 4294967982 rc 0/0 00000100:00100000:1.0:1789762898.821680:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 776 00000100:00100000:1.0:1789762898.821688:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372057224045 time 372057318234 state 0x2 ref 0 00000100:00100000:3.0:1789762898.823889:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762898.823956:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067490816 00000100:00100000:1.0:1789762898.823964:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067490816 00000100:00100000:1.0:1789762898.824001:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372057224045 time 372057318234 state 0x0 ref 1 00000100:00100000:1.0:1789762898.824012:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372157318234 time 372057318234 state 0x1 ref 1 00000100:00100000:1.0:1789762898.910322:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372157318234 time 372157388417 state 0x0 ref 1 00000100:00100000:1.0:1789762898.910347:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067490816:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762898.910394:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 55574528, mtime: 0x6aad9d52, atime: 0x0, ctime: 0x6aad9d52, blocks: 0x1a809 00010000:00010000:1.0:1789762898.910402:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176da7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.910419:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=372s v=(0 0) 00010000:00010000:1.0:1789762898.910423:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762898.910428:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176da7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.910439:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176da7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762898.922982:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967983 is committed 00002000:00010000:0.0:1789762898.922989:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 55574528 -> 56623104 00002000:00010000:0.0:1789762898.922994:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 108553 -> 110601 00010000:00010000:0.0:1789762898.923004:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176da7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.923014:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176da7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.923021:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176da7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.923033:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762898.923039:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176da7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.923049:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176da7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.923069:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176da7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.923078:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176da7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762898.923157:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067490816:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12819us (99268us total) trans 4294967983 rc 0/0 00000100:00100000:0.0:1789762898.923165:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 777 00000100:00100000:0.0:1789762898.923174:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372157318234 time 372157388417 state 0x2 ref 0 00000100:00100000:3.0:1789762898.925393:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762898.925466:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067490944 00000100:00100000:1.0:1789762898.925475:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067490944 00000100:00100000:1.0:1789762898.925510:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372157318234 time 372157388417 state 0x0 ref 1 00000100:00100000:1.0:1789762898.925522:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372257388417 time 372157388417 state 0x1 ref 1 00000004:00080000:3.0:1789762898.954681:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 19 committed 00000020:00080000:3.0:1789762898.954702:0:7221:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 11 batchid updated 00000004:00080000:3.0:1789762898.954712:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 19 committed 00000020:00080000:3.0:1789762898.954725:0:7221:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 12 batchid updated 00000020:00080000:1.0:1789762898.954742:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 12 00000100:00100000:3.0:1789762898.954791:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bdc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074721920:0@lo:6:osp-syn-1-1.0 00000004:00080000:1.0:1789762898.954816:0:7233:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c36c9a9d000 with version 26 00000020:00080000:1.0:1789762898.954829:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 11 cancel update log [0x200000401:0x2:0x0].11: rc = 0 00000100:00100000:3.0:1789762898.954859:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762898.954882:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c96a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074722048:0@lo:6:osp-syn-0-1.0 00000004:00080000:0.0:1789762898.954898:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 26 rpc_version 26 00000020:00100000:0.0:1789762898.954946:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762898.954956:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37d8ccf800 x1876702074722304/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1789762898.954957:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762898.954968:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074721920 00000100:00100000:0.0:1789762898.954977:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccf800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074722304:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762898.954977:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074721920 00000100:00100000:3.0:1789762898.954991:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074722048 00000100:00100000:3.0:1789762898.954997:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074722048 00000100:00100000:0.0:1789762898.955036:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762898.955038:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:3.0:1789762898.955047:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:2.0:1789762898.955052:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074721920:12345-0@lo:6:osp-syn-1-1.0 00000100:00100000:3.0:1789762898.955060:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074722048:12345-0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1789762898.955066:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37e1bc5300 going to sleep for 11 seconds 00002000:00080000:2.0:1789762898.955075:0:8054:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:12 count 1 00002000:00080000:3.0:1789762898.955076:0:8053:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:12 count 1 00010000:00010000:3.0:1789762898.955097:0:8053:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 3/0,1 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762898.955121:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000400:0xc:0x0] initial LVB size: 8388608, mtime: 0x6aad9d4c, atime: 0x0, ctime: 0x6aad9d4c, blocks: 0x4007 00010000:00010000:2.0:1789762898.955128:0:8054:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005a140c8b/0xdf5a7ac59f176db5 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762898.955132:0:8053:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a7b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9f9 expref: 6 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.955139:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xc:0x0].0x0 since=372s v=(0 0) 00010000:00010000:2.0:1789762898.955149:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762898.955159:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=6s v=(0 0) 00010000:00010000:2.0:1789762898.955160:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005a140c8b/0xdf5a7ac59f176db5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762898.955168:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762898.955175:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 4/0,1 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.955179:0:8054:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005a140c8b/0xdf5a7ac59f176db5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.955195:0:8054:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005a140c8b/0xdf5a7ac59f176db5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.955202:0:8054:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005a140c8b/0xdf5a7ac59f176db5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762898.955216:0:8053:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a7b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9f9 expref: 6 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.955219:0:8054:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762898.955235:0:8053:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a7b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9f9 expref: 6 pid: 9037 timeout: 472 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.955241:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1789762898.955247:0:8053:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c9ffa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074722432:192.168.203.37@tcp:104: 00010000:00010000:0.0:1789762898.955251:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005a140c8b/0xdf5a7ac59f176db5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.955264:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762898.955270:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005a140c8b/0xdf5a7ac59f176db5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.955280:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005a140c8b/0xdf5a7ac59f176db5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.955300:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005a140c8b/0xdf5a7ac59f176db5 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762898.955305:0:8053:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ca06e9c0 going to sleep for 11 seconds 00010000:00010000:0.0:1789762898.955310:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005a140c8b/0xdf5a7ac59f176db5 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762898.955330:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762898.955332:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762898.955334:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:1.0:1789762898.955596:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 11 cancel update log [0x240000400:0x2:0x0].11: rc = 0 00000004:00080000:1.0:1789762898.955625:0:7233:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c36c9a9d600 with version 27 00000020:00080000:1.0:1789762898.955629:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 12 cancel update log [0x200000401:0x2:0x0].12: rc = 0 00000020:00080000:1.0:1789762898.955852:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 12 cancel update log [0x240000400:0x2:0x0].12: rc = 0 00000020:00080000:1.0:1789762898.955860:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 12 00000100:00100000:1.0:1789762898.955879:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074722304 00000100:00100000:1.0:1789762898.955882:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074722304 00000100:00100000:1.0:1789762898.955945:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:1.0:1789762898.955955:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:7224:x1876702074722304:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762898.955988:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074721920:12345-0@lo:6:osp-syn-1-1.0 Request processed in 938us (1131us total) trans 4294967417 rc 0/0 00000100:00100000:2.0:1789762898.955997:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:0.0:1789762898.956011:0:3424:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294967416 gen 2 00000004:00080000:0.0:1789762898.956017:0:3424:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37f6089180, transno 4294967416 00000100:00080000:2.0:1789762898.956025:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762898.956030:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762898.956031:0:3424:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9c37e3bdc000 x1876702074721920/t4294967417(4294967417) o6->lustre-OST0001-osc-MDT0001@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789762914 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-1.0' uid:0 gid:0 projid:4294967295 00000004:00080000:0.0:1789762898.956042:0:3424:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37e3bdc000/2, rc 0, transno 4294967417 00000100:00100000:0.0:1789762898.956048:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bdc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074721920:0@lo:6:osp-syn-1-1.0 00000001:00080000:0.0:1789762898.956221:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967432 is committed 00010000:00010000:2.0:1789762898.956252:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956260:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956271:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956276:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5861a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762898.956279:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956285:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.956295:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074722304:12345-0@lo:1000:osp_up0-1.0 Request processed in 340us (1260us total) trans 4294967433 rc 0/0 00010000:00010000:2.0:1789762898.956299:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72bab00) 00010000:00010000:2.0:1789762898.956304:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762898.956307:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.956308:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00010000:00010000:2.0:1789762898.956316:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5861a00/0xdf5a7ac59f176b5b lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762898.956321:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccf800 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074722304:0@lo:1000:osp_up0-1.0 00000004:00080000:0.0:1789762898.956336:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 27 rpc_version 27 00010000:00010000:2.0:1789762898.956347:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860e00/0xdf5a7ac59f176b62 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1789762898.956351:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00010000:00010000:2.0:1789762898.956354:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860e00/0xdf5a7ac59f176b62 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762898.956360:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37d8ccca80 x1876702074722688/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1789762898.956362:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860e00/0xdf5a7ac59f176b62 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956378:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762898.956383:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860e00/0xdf5a7ac59f176b62 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762898.956390:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccca80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074722688:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1789762898.956403:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860e00/0xdf5a7ac59f176b62 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956415:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762898.956418:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762898.956424:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789762898.956425:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860e00/0xdf5a7ac59f176b62 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956435:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860e00/0xdf5a7ac59f176b62 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762898.956444:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37e1bc5300 going to sleep for 11 seconds 00000100:00100000:1.0:1789762898.956455:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074722688 00010000:00010000:2.0:1789762898.956456:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176bbd lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.956461:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074722688 00010000:00010000:2.0:1789762898.956464:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176bbd lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956470:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176bbd lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956474:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762898.956478:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176bbd lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956482:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176bbd lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956490:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762898.956493:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789762898.956496:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176bbd lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.956501:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 377 00000100:00100000:1.0:1789762898.956514:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074722688:12345-0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1789762898.956515:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860400/0xdf5a7ac59f176bbd lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956533:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176be0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956537:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176be0 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956542:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176be0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956546:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762898.956549:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176be0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956553:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176be0 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956562:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762898.956564:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762898.956567:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176be0 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956571:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f176be0 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956581:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176c11 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956586:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176c11 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956595:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176c11 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956599:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762898.956602:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176c11 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956606:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176c11 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956612:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x4009f790 (ffff9c37e37a7600) 00010000:00010000:2.0:1789762898.956614:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762898.956617:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176c11 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.956620:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f176c11 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762898.956639:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:7224:x1876702074722688:12345-0@lo:1000:osp_up0-1.0 Request processed in 128us (214us total) trans 4294967434 rc 0/0 00000100:00100000:1.0:1789762898.956647:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 377 00000100:00100000:0.0:1789762898.956662:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccca80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074722688:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1789762898.957074:0:8053:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c9ffa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8053:1876702074722432:192.168.203.37@tcp:104: 00010000:00010000:3.0:1789762898.957086:0:8053:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 3/0,1 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762898.957334:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762898.957366:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067491072 00000100:00100000:2.0:1789762898.957371:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067491072 00010000:00010000:2.0:1789762898.957389:0:9878:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a7b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9f9 expref: 7 pid: 9037 timeout: 472 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762898.957417:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 87 00000100:00100000:2.0:1789762898.957427:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffa680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067491072:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762898.957438:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762898.957445:0:9878:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a7b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9f9 expref: 7 pid: 9037 timeout: 472 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.957469:0:9878:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a7b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9f9 expref: 7 pid: 9037 timeout: 472 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.957482:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9476ce6/0xdf5a7ac59f176a7b lrc: 0/0,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9c9f9 expref: 7 pid: 9037 timeout: 472 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.957490:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762898.957492:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762898.957496:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.957502:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762898.957518:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762898.957542:0:8053:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762898.957545:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffa680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067491072:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 121us (211us total) trans 0 rc 0/0 00010000:00010000:3.0:1789762898.957561:0:8053:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762898.957562:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 87 00010000:00010000:3.0:1789762898.957589:0:8053:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762898.957598:0:8053:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762898.957619:0:8053:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762898.957642:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762898.957650:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.957661:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762898.957664:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.957671:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.957681:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 1/0,0 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.957691:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aeb3b0e2/0xdf5a7ac59f176dae lrc: 0/0,0 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762898.957704:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762898.957706:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762898.957708:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762898.958080:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074722048:12345-0@lo:6:osp-syn-0-1.0 Request processed in 3024us (3122us total) trans 4294967984 rc 0/0 00000100:00100000:3.0:1789762898.958086:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000004:00080000:1.0:1789762898.958093:0:3425:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37c96a0380/2, rc 0, transno 4294967984 00000100:00100000:1.0:1789762898.958100:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c96a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074722048:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1789762899.010399:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372257388417 time 372257468080 state 0x0 ref 1 00000100:00100000:0.0:1789762899.010426:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067490944:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762899.010459:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 56623104, mtime: 0x6aad9d52, atime: 0x0, ctime: 0x6aad9d52, blocks: 0x1b009 00010000:00010000:0.0:1789762899.010465:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dbc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.010477:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=372s v=(0 0) 00010000:00010000:0.0:1789762899.010480:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762899.010484:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789762899.010485:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762899.010497:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762899.022543:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967985 is committed 00000001:00080000:2.0:1789762899.022555:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967984 is committed 00002000:00010000:0.0:1789762899.022596:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 56623104 -> 57671680 00002000:00010000:0.0:1789762899.022600:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 110601 -> 112649 00010000:00010000:0.0:1789762899.022609:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.022618:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dbc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.022624:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.022629:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762899.022632:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.022639:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dbc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.022651:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dbc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.022656:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dbc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762899.022712:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067490944:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12299us (97319us total) trans 4294967985 rc 0/0 00000100:00100000:0.0:1789762899.022720:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 778 00000100:00100000:0.0:1789762899.022729:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372257388417 time 372257468080 state 0x2 ref 0 00000100:00100000:3.0:1789762899.025130:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762899.025192:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067491200 00000100:00100000:1.0:1789762899.025201:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067491200 00000100:00100000:1.0:1789762899.025237:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372257388417 time 372257468080 state 0x0 ref 1 00000100:00100000:1.0:1789762899.025249:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372357468080 time 372257468080 state 0x1 ref 1 00000100:00100000:0.0:1789762899.110477:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372357468080 time 372357543607 state 0x0 ref 1 00000100:00100000:0.0:1789762899.110504:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067491200:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762899.110546:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 57671680, mtime: 0x6aad9d52, atime: 0x0, ctime: 0x6aad9d52, blocks: 0x1b809 00010000:00010000:0.0:1789762899.110554:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dc3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.110567:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=372s v=(0 0) 00010000:00010000:0.0:1789762899.110571:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762899.110575:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.110586:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762899.121496:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967986 is committed 00002000:00010000:0.0:1789762899.121548:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 57671680 -> 58720256 00002000:00010000:0.0:1789762899.121553:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 112649 -> 114697 00010000:00010000:0.0:1789762899.121562:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.121573:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.121589:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.121597:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762899.121602:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.121612:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dc3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.121632:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dc3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.121639:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dc3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762899.121698:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067491200:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11200us (96567us total) trans 4294967986 rc 0/0 00000100:00100000:0.0:1789762899.121706:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 779 00000100:00100000:0.0:1789762899.121714:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372357468080 time 372357543607 state 0x2 ref 0 00000100:00100000:3.0:1789762899.124039:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762899.124068:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067491328 00000100:00100000:0.0:1789762899.124075:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067491328 00000100:00100000:0.0:1789762899.124122:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372357468080 time 372357543607 state 0x0 ref 1 00000100:00100000:0.0:1789762899.124135:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372457543607 time 372357543607 state 0x1 ref 1 00000100:00100000:0.0:1789762899.210524:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372457543607 time 372457590040 state 0x0 ref 1 00000100:00100000:0.0:1789762899.210544:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ff9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067491328:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762899.210592:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 58720256, mtime: 0x6aad9d52, atime: 0x0, ctime: 0x6aad9d52, blocks: 0x1c009 00010000:00010000:0.0:1789762899.210602:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.210619:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=372s v=(0 0) 00010000:00010000:0.0:1789762899.210623:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762899.210628:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.210640:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762899.221312:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967987 is committed 00002000:00010000:0.0:1789762899.221340:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 58720256 -> 59768832 00002000:00010000:0.0:1789762899.221345:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 114697 -> 116745 00010000:00010000:0.0:1789762899.221354:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.221363:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.221374:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.221379:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762899.221382:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.221388:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.221402:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.221406:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762899.221462:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ff9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067491328:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10924us (97423us total) trans 4294967987 rc 0/0 00000100:00100000:0.0:1789762899.221471:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 780 00000100:00100000:0.0:1789762899.221480:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372457543607 time 372457590040 state 0x2 ref 0 00000100:00100000:3.0:1789762899.223673:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762899.223711:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067491456 00000100:00100000:0.0:1789762899.223719:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067491456 00000100:00100000:0.0:1789762899.223754:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372457543607 time 372457590040 state 0x0 ref 1 00000100:00100000:0.0:1789762899.223768:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372557590040 time 372457590040 state 0x1 ref 1 00000100:00100000:0.0:1789762899.310568:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372557590040 time 372557637527 state 0x0 ref 1 00000100:00100000:0.0:1789762899.310585:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067491456:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762899.310625:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 59768832, mtime: 0x6aad9d52, atime: 0x0, ctime: 0x6aad9d52, blocks: 0x1c809 00010000:00010000:0.0:1789762899.310634:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.310650:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=372s v=(0 0) 00010000:00010000:0.0:1789762899.310655:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762899.310662:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.310673:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762899.321054:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967988 is committed 00002000:00010000:1.0:1789762899.321082:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 59768832 -> 60817408 00002000:00010000:1.0:1789762899.321086:0:14171:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb mtime from disk: 1789762898 -> 1789762899 00002000:00010000:1.0:1789762899.321089:0:14171:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb ctime from disk: 1789762898 -> 1789762899 00002000:00010000:1.0:1789762899.321090:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 116745 -> 118793 00010000:00010000:1.0:1789762899.321099:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.321124:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.321134:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.321143:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762899.321148:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.321159:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dd1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.321178:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dd1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.321184:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f176dd1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762899.321237:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067491456:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10656us (97565us total) trans 4294967988 rc 0/0 00000100:00100000:1.0:1789762899.321245:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 781 00000100:00100000:1.0:1789762899.321253:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372557590040 time 372557637527 state 0x2 ref 0 00000100:00100000:3.0:1789762899.323343:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762899.323391:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067491584 00000100:00100000:0.0:1789762899.323400:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067491584 00000100:00100000:0.0:1789762899.323446:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372557590040 time 372557637527 state 0x0 ref 1 00000100:00100000:0.0:1789762899.323457:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372657637527 time 372557637527 state 0x1 ref 1 00000100:00100000:2.0:1789762899.399843:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762899.399900:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067491712 00000100:00100000:1.0:1789762899.399907:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067491712 00000100:00100000:2.0:1789762899.399946:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762899.399985:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 371 00000100:00100000:1.0:1789762899.399998:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1551450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067491712:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762899.400005:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067491968 00000100:00100000:2.0:1789762899.400005:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762899.400012:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067491968 00000100:00100000:3.0:1789762899.400034:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067492224 00000100:00100000:2.0:1789762899.400035:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762899.400043:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067492224 00000100:00100000:1.0:1789762899.400065:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1551450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067491712:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 70us (227us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762899.400066:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067491840 00000100:00100000:0.0:1789762899.400073:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067491840 00000100:00100000:1.0:1789762899.400080:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 371 00000100:00100000:3.0:1789762899.400118:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 548 00000100:00100000:3.0:1789762899.400133:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2528380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067491968:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762899.400142:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 549 00000100:00100000:1.0:1789762899.400152:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762899.400158:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067492224:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762899.400164:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 550 00000100:00100000:2.0:1789762899.400175:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067491840:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762899.400230:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2528380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067491968:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 99us (308us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762899.400232:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067492224:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 81us (233us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762899.400237:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067491840:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 61us (199us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762899.400247:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762899.400253:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 548 00000100:00100000:0.0:1789762899.400260:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 549 00000100:00100000:2.0:1789762899.400267:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 550 00000100:00100000:3.0:1789762899.400277:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067492352 00000100:00100000:3.0:1789762899.400283:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067492352 00000100:00080000:0.0:1789762899.400289:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762899.400294:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762899.400327:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 551 00000100:00100000:3.0:1789762899.400340:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067492352:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762899.400343:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067492096 00000100:00100000:1.0:1789762899.400348:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067492096 00000100:00080000:0.0:1789762899.400383:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762899.400390:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 224 00000100:00100000:1.0:1789762899.400400:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067492096:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762899.400449:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067492096:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 46us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762899.400450:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067492352:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 108us (204us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762899.400460:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 224 00000100:00100000:3.0:1789762899.400469:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 551 00000100:00100000:1.0:1789762899.410737:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372657637527 time 372657793275 state 0x0 ref 1 00000100:00100000:1.0:1789762899.410771:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067491584:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762899.410836:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 60817408, mtime: 0x6aad9d53, atime: 0x0, ctime: 0x6aad9d53, blocks: 0x1d009 00010000:00010000:1.0:1789762899.410849:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176dd8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.410872:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=372s v=(0 0) 00010000:00010000:1.0:1789762899.410878:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762899.410886:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176dd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.410905:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176dd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762899.428616:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967989 is committed 00002000:00010000:2.0:1789762899.428741:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 60817408 -> 61865984 00002000:00010000:2.0:1789762899.428754:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 118793 -> 120841 00010000:00010000:2.0:1789762899.428774:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176dd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.428799:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176dd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.428819:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176dd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.428836:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762899.428846:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176dd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.428866:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176dd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.428899:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176dd8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.428972:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e880cc5/0xdf5a7ac59f176dd8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762899.429078:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f6089180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067491584:12345-192.168.203.37@tcp:4:dd.500 Request processed in 18314us (105730us total) trans 4294967989 rc 0/0 00000100:00100000:2.0:1789762899.429092:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 782 00000100:00100000:2.0:1789762899.429129:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372657637527 time 372657793275 state 0x2 ref 0 00000100:00100000:2.0:1789762899.432446:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762899.432533:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067492480 00000100:00100000:0.0:1789762899.432546:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067492480 00000100:00100000:0.0:1789762899.432602:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372657637527 time 372657793275 state 0x0 ref 1 00000100:00100000:0.0:1789762899.432628:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372757793275 time 372657793275 state 0x1 ref 1 00000100:00100000:1.0:1789762899.510865:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372757793275 time 372757924617 state 0x0 ref 1 00000100:00100000:1.0:1789762899.510897:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067492480:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762899.511008:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 61865984, mtime: 0x6aad9d53, atime: 0x0, ctime: 0x6aad9d53, blocks: 0x1d809 00010000:00010000:1.0:1789762899.511021:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176ddf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.511043:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=372s v=(0 0) 00010000:00010000:1.0:1789762899.511049:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762899.511057:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176ddf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.511074:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176ddf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762899.533505:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967990 is committed 00002000:00010000:2.0:1789762899.533545:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 61865984 -> 62914560 00002000:00010000:2.0:1789762899.533556:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 120841 -> 122889 00010000:00010000:2.0:1789762899.533572:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176ddf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.533589:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176ddf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.533601:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176ddf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.533610:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762cc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762899.533616:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176ddf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.533627:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176ddf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.533666:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176ddf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.533676:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003d83967f/0xdf5a7ac59f176ddf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762899.533787:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067492480:12345-192.168.203.37@tcp:4:dd.500 Request processed in 22895us (101333us total) trans 4294967990 rc 0/0 00000100:00100000:2.0:1789762899.533807:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 783 00000100:00100000:2.0:1789762899.533830:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372757793275 time 372757924617 state 0x2 ref 0 00000100:00100000:2.0:1789762899.537968:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762899.538060:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067492608 00000100:00100000:1.0:1789762899.538074:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067492608 00000100:00100000:1.0:1789762899.538175:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372757793275 time 372757924617 state 0x0 ref 1 00000100:00100000:1.0:1789762899.538204:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372857924617 time 372757924617 state 0x1 ref 1 00010000:00010000:2.0:1789762899.584078:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762899.611019:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372857924617 time 372858073939 state 0x0 ref 1 00000100:00100000:1.0:1789762899.611056:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067492608:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762899.611155:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 62914560, mtime: 0x6aad9d53, atime: 0x0, ctime: 0x6aad9d53, blocks: 0x1e009 00010000:00010000:1.0:1789762899.611174:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176de6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.611203:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=372s v=(0 0) 00010000:00010000:1.0:1789762899.611209:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762899.611217:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176de6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.611236:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176de6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762899.633281:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967991 is committed 00002000:00010000:1.0:1789762899.633340:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 62914560 -> 63963136 00002000:00010000:1.0:1789762899.633350:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 122889 -> 124937 00010000:00010000:1.0:1789762899.633366:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176de6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.633383:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176de6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.633395:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176de6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.633404:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762dc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762899.633411:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176de6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.633424:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176de6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.633447:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176de6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.633456:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176de6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762899.633557:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067492608:12345-192.168.203.37@tcp:4:dd.500 Request processed in 22511us (95636us total) trans 4294967991 rc 0/0 00000100:00100000:1.0:1789762899.633571:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 784 00000100:00100000:1.0:1789762899.633586:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372857924617 time 372858073939 state 0x2 ref 0 00000100:00100000:0.0:1789762899.637799:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762899.637902:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067492736 00000100:00100000:2.0:1789762899.637946:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067492736 00000100:00100000:2.0:1789762899.638015:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372857924617 time 372858073939 state 0x0 ref 1 00000100:00100000:2.0:1789762899.638035:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372958073939 time 372858073939 state 0x1 ref 1 00000100:00100000:1.0:1789762899.711215:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372958073939 time 372958271684 state 0x0 ref 1 00000100:00100000:1.0:1789762899.711248:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067492736:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762899.711301:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 63963136, mtime: 0x6aad9d53, atime: 0x0, ctime: 0x6aad9d53, blocks: 0x1e809 00010000:00010000:1.0:1789762899.711313:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ded lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.711333:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=372s v=(0 0) 00010000:00010000:1.0:1789762899.711340:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762899.711348:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ded lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762899.711365:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ded lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762899.732683:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967992 is committed 00002000:00010000:2.0:1789762899.732831:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 63963136 -> 65011712 00002000:00010000:2.0:1789762899.732841:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 124937 -> 126985 00010000:00010000:2.0:1789762899.732856:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ded lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.732893:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ded lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.732905:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ded lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.732914:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762dc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762899.732952:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ded lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.732965:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ded lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.732987:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ded lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.732997:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ded lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762899.733090:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067492736:12345-192.168.203.37@tcp:4:dd.500 Request processed in 21849us (95292us total) trans 4294967992 rc 0/0 00000100:00100000:2.0:1789762899.733134:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 785 00000100:00100000:2.0:1789762899.733150:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372958073939 time 372958271684 state 0x2 ref 0 00000100:00100000:2.0:1789762899.737067:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762899.737172:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067492864 00000100:00100000:3.0:1789762899.737186:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067492864 00000100:00100000:3.0:1789762899.737244:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 372958073939 time 372958271684 state 0x0 ref 1 00000100:00100000:3.0:1789762899.737263:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373058271684 time 372958271684 state 0x1 ref 1 00000100:00100000:0.0:1789762899.811386:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373058271684 time 373058444619 state 0x0 ref 1 00000100:00100000:0.0:1789762899.811422:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067492864:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762899.811493:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 65011712, mtime: 0x6aad9d53, atime: 0x0, ctime: 0x6aad9d53, blocks: 0x1f009 00010000:00010000:0.0:1789762899.811505:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176df4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.811526:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=373s v=(0 0) 00010000:00010000:0.0:1789762899.811534:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762899.811545:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176df4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.811576:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176df4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762899.833233:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967993 is committed 00002000:00010000:2.0:1789762899.833326:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 65011712 -> 66060288 00002000:00010000:2.0:1789762899.833336:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 126985 -> 129033 00010000:00010000:2.0:1789762899.833353:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176df4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.833370:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176df4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.833383:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176df4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.833392:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762899.833398:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176df4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.833410:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176df4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.833432:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176df4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762899.833441:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176df4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762899.833553:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067492864:12345-192.168.203.37@tcp:4:dd.500 Request processed in 22140us (96485us total) trans 4294967993 rc 0/0 00000100:00100000:2.0:1789762899.833567:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 786 00000100:00100000:2.0:1789762899.833597:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373058271684 time 373058444619 state 0x2 ref 0 00000100:00100000:3.0:1789762899.838087:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762899.838206:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067492992 00000100:00100000:1.0:1789762899.838220:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067492992 00000100:00100000:1.0:1789762899.838291:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373058271684 time 373058444619 state 0x0 ref 1 00000100:00100000:1.0:1789762899.838320:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373158444619 time 373058444619 state 0x1 ref 1 00000100:00100000:0.0:1789762899.911557:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373158444619 time 373158607331 state 0x0 ref 1 00000100:00100000:0.0:1789762899.911603:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067492992:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762899.911684:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 66060288, mtime: 0x6aad9d53, atime: 0x0, ctime: 0x6aad9d53, blocks: 0x1f809 00010000:00010000:0.0:1789762899.911696:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176dfb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.911718:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=373s v=(0 0) 00010000:00010000:0.0:1789762899.911725:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762899.911733:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176dfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.911751:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176dfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762899.930589:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 66060288 -> 67108864 00002000:00010000:0.0:1789762899.930599:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 129033 -> 131081 00010000:00010000:0.0:1789762899.930615:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176dfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.930641:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176dfb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.930654:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176dfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762899.930660:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967994 is committed 00010000:00010000:0.0:1789762899.930665:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762899.930675:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176dfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.930687:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176dfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.930710:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176dfb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762899.930721:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176dfb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762899.930835:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067492992:12345-192.168.203.37@tcp:4:dd.500 Request processed in 19249us (92750us total) trans 4294967994 rc 0/0 00000100:00100000:0.0:1789762899.930846:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 787 00000100:00100000:0.0:1789762899.930858:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373158444619 time 373158607331 state 0x2 ref 0 00000100:00100000:3.0:1789762899.934757:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762899.934831:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067493120 00000100:00100000:2.0:1789762899.934842:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067493120 00000100:00100000:2.0:1789762899.934901:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373158444619 time 373158607331 state 0x0 ref 1 00000100:00100000:2.0:1789762899.934949:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373258607331 time 373158607331 state 0x1 ref 1 00000100:00100000:1.0:1789762900.011698:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373258607331 time 373258757681 state 0x0 ref 1 00000100:00100000:1.0:1789762900.011732:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067493120:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762900.011793:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 67108864, mtime: 0x6aad9d53, atime: 0x0, ctime: 0x6aad9d53, blocks: 0x20009 00010000:00010000:1.0:1789762900.011804:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176e02 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762900.011823:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=373s v=(0 0) 00010000:00010000:1.0:1789762900.011830:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762900.011840:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176e02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762900.011893:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176e02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762900.033905:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 67108864 -> 68157440 00002000:00010000:0.0:1789762900.033949:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 131081 -> 133129 00000001:00080000:2.0:1789762900.033949:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967995 is committed 00010000:00010000:0.0:1789762900.033974:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176e02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.033994:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176e02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.034009:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176e02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.034018:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762d400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762900.034023:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176e02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.034032:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176e02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.034053:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176e02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.034061:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176e02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762900.034202:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067493120:12345-192.168.203.37@tcp:4:dd.500 Request processed in 22474us (99459us total) trans 4294967995 rc 0/0 00000100:00100000:0.0:1789762900.034222:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 788 00000100:00100000:0.0:1789762900.034243:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373258607331 time 373258757681 state 0x2 ref 0 00000100:00100000:3.0:1789762900.038348:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762900.038472:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067493248 00000100:00100000:2.0:1789762900.038483:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067493248 00000100:00100000:2.0:1789762900.038539:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373258607331 time 373258757681 state 0x0 ref 1 00000100:00100000:2.0:1789762900.038563:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373358757681 time 373258757681 state 0x1 ref 1 00000100:00100000:2.0:1789762900.111827:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373358757681 time 373358883024 state 0x0 ref 1 00000100:00100000:2.0:1789762900.111863:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067493248:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762900.111968:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 68157440, mtime: 0x6aad9d53, atime: 0x0, ctime: 0x6aad9d53, blocks: 0x20809 00010000:00010000:2.0:1789762900.111981:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.112003:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=373s v=(0 0) 00010000:00010000:2.0:1789762900.112009:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762900.112017:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.112057:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762900.135248:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 68157440 -> 69206016 00000001:00080000:0.0:1789762900.135256:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967996 is committed 00002000:00010000:2.0:1789762900.135262:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 133129 -> 135177 00010000:00010000:2.0:1789762900.135285:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.135303:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.135317:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.135326:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762900.135332:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.135343:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.135367:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.135376:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762900.135505:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067493248:12345-192.168.203.37@tcp:4:dd.500 Request processed in 23649us (97155us total) trans 4294967996 rc 0/0 00000100:00100000:2.0:1789762900.135520:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 789 00000100:00100000:2.0:1789762900.135537:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373358757681 time 373358883024 state 0x2 ref 0 00000100:00100000:3.0:1789762900.140002:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762900.140129:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067493376 00000100:00100000:0.0:1789762900.140142:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067493376 00000100:00100000:0.0:1789762900.140206:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373358757681 time 373358883024 state 0x0 ref 1 00000100:00100000:0.0:1789762900.140226:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373458883024 time 373358883024 state 0x1 ref 1 00000100:00100000:3.0:1789762900.212015:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373458883024 time 373459064359 state 0x0 ref 1 00000100:00100000:3.0:1789762900.212061:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067493376:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762900.212174:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 69206016, mtime: 0x6aad9d53, atime: 0x0, ctime: 0x6aad9d53, blocks: 0x21009 00010000:00010000:3.0:1789762900.212191:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006311a7bc/0xdf5a7ac59f176e10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762900.212221:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=373s v=(0 0) 00010000:00010000:3.0:1789762900.212231:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762900.212243:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006311a7bc/0xdf5a7ac59f176e10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762900.212306:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006311a7bc/0xdf5a7ac59f176e10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762900.232737:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967997 is committed 00002000:00010000:2.0:1789762900.232766:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 69206016 -> 70254592 00002000:00010000:2.0:1789762900.232771:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 135177 -> 137225 00010000:00010000:2.0:1789762900.232781:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006311a7bc/0xdf5a7ac59f176e10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.232792:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006311a7bc/0xdf5a7ac59f176e10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.232798:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006311a7bc/0xdf5a7ac59f176e10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.232803:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762900.232807:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006311a7bc/0xdf5a7ac59f176e10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.232814:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006311a7bc/0xdf5a7ac59f176e10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.232828:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006311a7bc/0xdf5a7ac59f176e10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.232845:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006311a7bc/0xdf5a7ac59f176e10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762900.232913:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067493376:12345-192.168.203.37@tcp:4:dd.500 Request processed in 20870us (92919us total) trans 4294967997 rc 0/0 00000100:00100000:2.0:1789762900.232948:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 790 00000100:00100000:2.0:1789762900.232958:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373458883024 time 373459064359 state 0x2 ref 0 00000100:00100000:3.0:1789762900.235291:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762900.235338:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067493504 00000100:00100000:0.0:1789762900.235346:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067493504 00000100:00100000:0.0:1789762900.235384:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373458883024 time 373459064359 state 0x0 ref 1 00000100:00100000:0.0:1789762900.235395:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373559064359 time 373459064359 state 0x1 ref 1 00000100:00100000:1.0:1789762900.312080:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373559064359 time 373559145756 state 0x0 ref 1 00000100:00100000:1.0:1789762900.312115:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067493504:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762900.312162:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 70254592, mtime: 0x6aad9d53, atime: 0x0, ctime: 0x6aad9d53, blocks: 0x21809 00010000:00010000:1.0:1789762900.312169:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176e17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762900.312181:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=373s v=(0 0) 00010000:00010000:1.0:1789762900.312185:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762900.312190:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176e17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762900.312201:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176e17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762900.323549:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967998 is committed 00002000:00010000:2.0:1789762900.323577:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 70254592 -> 71303168 00002000:00010000:2.0:1789762900.323582:0:8057:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb mtime from disk: 1789762899 -> 1789762900 00002000:00010000:2.0:1789762900.323584:0:8057:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb ctime from disk: 1789762899 -> 1789762900 00002000:00010000:2.0:1789762900.323586:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 137225 -> 139273 00010000:00010000:2.0:1789762900.323596:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176e17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.323605:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176e17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.323612:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176e17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.323617:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762de00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762900.323621:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176e17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.323638:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176e17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.323653:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176e17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.323658:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf9e174b/0xdf5a7ac59f176e17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762900.323723:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067493504:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11626us (88430us total) trans 4294967998 rc 0/0 00000100:00100000:2.0:1789762900.323731:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 791 00000100:00100000:2.0:1789762900.323740:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373559064359 time 373559145756 state 0x2 ref 0 00000100:00100000:3.0:1789762900.326174:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762900.326209:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067493632 00000100:00100000:2.0:1789762900.326217:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067493632 00000100:00100000:2.0:1789762900.326253:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373559064359 time 373559145756 state 0x0 ref 1 00000100:00100000:2.0:1789762900.326263:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373659145756 time 373559145756 state 0x1 ref 1 00000100:00100000:2.0:1789762900.412156:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373659145756 time 373659218863 state 0x0 ref 1 00000100:00100000:2.0:1789762900.412178:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067493632:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762900.412225:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 71303168, mtime: 0x6aad9d54, atime: 0x0, ctime: 0x6aad9d54, blocks: 0x22009 00010000:00010000:2.0:1789762900.412244:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.412257:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=373s v=(0 0) 00010000:00010000:2.0:1789762900.412260:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762900.412265:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.412276:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762900.423069:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 71303168 -> 72351744 00000001:00080000:0.0:1789762900.423070:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967999 is committed 00002000:00010000:2.0:1789762900.423076:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 139273 -> 141321 00010000:00010000:2.0:1789762900.423087:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.423110:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.423119:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.423125:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762900.423128:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.423134:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.423155:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.423161:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f176e1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762900.423210:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067493632:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11039us (97036us total) trans 4294967999 rc 0/0 00000100:00100000:2.0:1789762900.423218:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 792 00000100:00100000:2.0:1789762900.423226:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373659145756 time 373659218863 state 0x2 ref 0 00000100:00100000:3.0:1789762900.425353:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762900.425384:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067493760 00000100:00100000:2.0:1789762900.425392:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067493760 00000100:00100000:2.0:1789762900.425430:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373659145756 time 373659218863 state 0x0 ref 1 00000100:00100000:2.0:1789762900.425444:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373759218863 time 373659218863 state 0x1 ref 1 00000100:00100000:1.0:1789762900.480092:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cbac4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074722816:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762900.480092:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074723072:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762900.480134:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff70dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074723200:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762900.480173:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762900.480180:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8ccd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074722944:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762900.480221:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762900.480242:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff698a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074723456:0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.480271:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff699f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074723712:0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.480278:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762900.480303:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762900.480303:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074723456 00000100:00100000:3.0:1789762900.480308:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff686d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074723968:0@lo:400:kworker.0 00000100:00100000:2.0:1789762900.480311:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074723456 00000100:00100000:1.0:1789762900.480331:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff686680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074724224:0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.480343:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762900.480366:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074723072 00000100:00100000:1.0:1789762900.480370:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074723072 00000100:00100000:1.0:1789762900.480402:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074722944 00000100:00100000:2.0:1789762900.480405:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074723712 00000100:00100000:0.0:1789762900.480406:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074723968 00000100:00100000:1.0:1789762900.480406:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074722944 00000100:00100000:2.0:1789762900.480413:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074723712 00000100:00100000:0.0:1789762900.480414:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074723968 00000100:00100000:3.0:1789762900.480435:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff685880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074724480:0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.480440:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:1.0:1789762900.480450:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074723072:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762900.480453:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 552 00000100:00100000:0.0:1789762900.480462:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 553 00000100:00100000:2.0:1789762900.480467:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074723456:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762900.480475:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074723712:12345-0@lo:400:kworker.0 00002000:00000024:1.0:1789762900.480481:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3851419648 avail 3849322496 00000100:00100000:2.0:1789762900.480518:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074723456:12345-0@lo:400:kworker.0 Request processed in 56us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762900.480534:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074723072:12345-0@lo:13:osp-pre-1-0.0 Request processed in 85us (362us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762900.480537:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 552 00000100:00100000:1.0:1789762900.480547:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00100000:0.0:1789762900.480549:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074723712:12345-0@lo:400:kworker.0 Request processed in 74us (243us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762900.480558:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 553 00000100:00100000:2.0:1789762900.480571:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 554 00000100:00100000:1.0:1789762900.480573:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:3.0:1789762900.480573:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762900.480588:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074722944:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762900.480592:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074723968:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762900.480597:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:1.0:1789762900.480603:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3851419648 avail 3849322496 00000100:00100000:0.0:1789762900.480642:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff69aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074723328:0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.480658:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074722944:12345-0@lo:13:osp-pre-1-1.0 Request processed in 73us (436us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762900.480662:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074723968:12345-0@lo:400:kworker.0 Request processed in 78us (317us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762900.480676:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074723072:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762900.480681:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00100000:2.0:1789762900.480683:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 554 00000100:00100000:0.0:1789762900.480687:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762900.480705:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074722816 00000100:00100000:1.0:1789762900.480709:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074722816 00000100:00100000:0.0:1789762900.480728:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff698e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074723840:0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.480731:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff698a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074723456:0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.480737:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:2.0:1789762900.480739:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762900.480750:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff687480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074722816:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762900.480760:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff686d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074723968:0@lo:400:kworker.0 00002000:00000024:1.0:1789762900.480761:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 525926400 pending 0 free 3794796544 avail 3785359360 00000100:00100000:2.0:1789762900.480769:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff69a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074723584:0@lo:400:kworker.0 00000100:00100000:0.0:1789762900.480773:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762900.480792:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074723200 00000100:00100000:3.0:1789762900.480797:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074723200 00000100:00100000:0.0:1789762900.480808:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff687800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074724352:0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.480811:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff687480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074722816:12345-0@lo:13:osp-pre-0-0.0 Request processed in 62us (217us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762900.480813:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762900.480823:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:3.0:1789762900.480834:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 229 00000100:00100000:3.0:1789762900.480845:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074723200:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762900.480850:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff686a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074724096:0@lo:400:kworker.0 00000100:00080000:1.0:1789762900.480852:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762900.480855:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:3.0:1789762900.480857:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 525926400 pending 0 free 3794796544 avail 3785359360 00000100:00080000:1.0:1789762900.480859:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762900.480866:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762900.480885:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074724480 00000100:00100000:0.0:1789762900.480886:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff684380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074724864:0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.480891:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074724480 00000100:00100000:2.0:1789762900.480894:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762900.480903:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074723200:12345-0@lo:13:osp-pre-0-1.0 Request processed in 59us (177us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762900.480910:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 229 00000100:00100000:2.0:1789762900.480945:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff685500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074724608:0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.480951:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:1.0:1789762900.480960:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074724480:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.480964:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074724352 00000100:00100000:0.0:1789762900.480966:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762900.480973:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074724352 00000100:00100000:2.0:1789762900.480986:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074723840 00000100:00100000:2.0:1789762900.480991:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074723840 00000100:00100000:0.0:1789762900.481020:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cbac4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074722816:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762900.481026:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074724480:12345-0@lo:400:kworker.0 Request processed in 65us (456us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762900.481028:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074724096 00000100:00100000:2.0:1789762900.481031:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074723584 00000100:00100000:3.0:1789762900.481035:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074724096 00000100:00100000:2.0:1789762900.481036:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074723584 00000100:00100000:1.0:1789762900.481042:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00100000:0.0:1789762900.481064:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074723328 00000100:00100000:1.0:1789762900.481065:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074724864 00000100:00100000:2.0:1789762900.481071:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 378 00000100:00100000:1.0:1789762900.481074:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074724864 00000100:00100000:0.0:1789762900.481075:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074723328 00000100:00100000:3.0:1789762900.481083:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 555 00000100:00100000:2.0:1789762900.481084:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074723840:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.481109:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074724352:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762900.481139:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:1.0:1789762900.481139:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:0.0:1789762900.481156:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1552850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074723328:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.481159:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074724864:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762900.481168:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074723840:12345-0@lo:400:kworker.0 Request processed in 86us (396us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762900.481168:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074724352:12345-0@lo:400:kworker.0 Request processed in 73us (313us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762900.481188:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 555 00000100:00100000:2.0:1789762900.481189:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 378 00000100:00100000:0.0:1789762900.481210:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1552850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074723328:12345-0@lo:400:kworker.0 Request processed in 57us (530us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762900.481212:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074724864:12345-0@lo:400:kworker.0 Request processed in 61us (248us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762900.481219:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 556 00000100:00100000:2.0:1789762900.481220:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 379 00000100:00100000:0.0:1789762900.481223:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00100000:1.0:1789762900.481229:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00100000:2.0:1789762900.481238:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074723584:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.481240:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074724096:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789762900.481259:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762900.481274:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762900.481294:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074724224 00000100:00100000:0.0:1789762900.481298:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074724224 00000100:00100000:3.0:1789762900.481303:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074724096:12345-0@lo:400:kworker.0 Request processed in 68us (411us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762900.481304:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074723584:12345-0@lo:400:kworker.0 Request processed in 71us (494us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762900.481306:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff685180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074724736:0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.481321:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 556 00000100:00100000:0.0:1789762900.481330:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 557 00000100:00100000:2.0:1789762900.481335:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 379 00000100:00100000:0.0:1789762900.481343:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cbac4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074724224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.481344:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762900.481384:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8ccd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074722944:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762900.481390:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cbac4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074724224:12345-0@lo:400:kworker.0 Request processed in 49us (114us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762900.481399:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 557 00000100:00100000:1.0:1789762900.481405:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff699f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074723712:0@lo:400:kworker.0 00000100:00100000:2.0:1789762900.481417:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff69aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074723328:0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.481423:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff686680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074724224:0@lo:400:kworker.0 00000100:00100000:2.0:1789762900.481439:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff698e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074723840:0@lo:400:kworker.0 00000100:00080000:0.0:1789762900.481447:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762900.481461:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff687800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074724352:0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.481473:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074724736 00000100:00100000:0.0:1789762900.481477:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294967984 gen 2 00000100:00100000:1.0:1789762900.481477:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074724736 00000100:00100000:2.0:1789762900.481482:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff684380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074724864:0@lo:400:kworker.0 00000004:00080000:0.0:1789762900.481485:0:3427:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37c96a0380, transno 4294967984 00000100:00100000:3.0:1789762900.481489:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762900.481505:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff685880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074724480:0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.481512:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074724608 00000100:00100000:1.0:1789762900.481515:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074724608 00000100:00100000:1.0:1789762900.481539:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:1.0:1789762900.481547:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074724736:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.481552:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff70dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074723200:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762900.481565:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff69a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074723584:0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.481591:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff686a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074724096:0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.481594:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074724736:12345-0@lo:400:kworker.0 Request processed in 46us (250us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762900.481600:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00100000:1.0:1789762900.481618:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:2.0:1789762900.481626:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff685180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074724736:0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.481631:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074724608:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.481677:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074724608:12345-0@lo:400:kworker.0 Request processed in 52us (193us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762900.481682:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:3.0:1789762900.481710:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff685500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074724608:0@lo:400:kworker.0 00000100:00100000:2.0:1789762900.512209:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373759218863 time 373759280431 state 0x0 ref 1 00000100:00100000:2.0:1789762900.512223:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067493760:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762900.512257:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 72351744, mtime: 0x6aad9d54, atime: 0x0, ctime: 0x6aad9d54, blocks: 0x22809 00010000:00010000:2.0:1789762900.512264:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.512277:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=373s v=(0 0) 00010000:00010000:2.0:1789762900.512280:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762900.512285:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.512295:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762900.523583:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 72351744 -> 73400320 00000001:00080000:0.0:1789762900.523584:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968000 is committed 00002000:00010000:2.0:1789762900.523591:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 141321 -> 143369 00010000:00010000:2.0:1789762900.523607:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.523616:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.523623:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.523628:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762900.523631:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.523638:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.523653:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.523661:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762900.523722:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067493760:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11500us (98367us total) trans 4294968000 rc 0/0 00000100:00100000:2.0:1789762900.523730:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 793 00000100:00100000:2.0:1789762900.523739:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373759218863 time 373759280431 state 0x2 ref 0 00000100:00100000:3.0:1789762900.525757:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762900.525791:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067493888 00000100:00100000:2.0:1789762900.525799:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067493888 00000100:00100000:2.0:1789762900.525832:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373759218863 time 373759280431 state 0x0 ref 1 00000100:00100000:2.0:1789762900.525842:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373859280431 time 373759280431 state 0x1 ref 1 00010000:00010000:3.0:1789762900.608047:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762900.612285:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373859280431 time 373859354949 state 0x0 ref 1 00000100:00100000:0.0:1789762900.612307:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067493888:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762900.612353:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 73400320, mtime: 0x6aad9d54, atime: 0x0, ctime: 0x6aad9d54, blocks: 0x23009 00010000:00010000:0.0:1789762900.612360:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.612373:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=373s v=(0 0) 00010000:00010000:0.0:1789762900.612377:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762900.612382:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.612393:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762900.623324:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 73400320 -> 74448896 00002000:00010000:0.0:1789762900.623329:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 143369 -> 145417 00000001:00080000:2.0:1789762900.623336:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968001 is committed 00010000:00010000:0.0:1789762900.623339:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.623350:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.623356:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.623366:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762900.623369:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.623376:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.623388:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.623393:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762900.623445:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067493888:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11145us (97687us total) trans 4294968001 rc 0/0 00000100:00100000:0.0:1789762900.623453:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 794 00000100:00100000:0.0:1789762900.623461:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373859280431 time 373859354949 state 0x2 ref 0 00000100:00100000:3.0:1789762900.625703:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762900.625754:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067494016 00000100:00100000:2.0:1789762900.625762:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067494016 00000100:00100000:2.0:1789762900.625799:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373859280431 time 373859354949 state 0x0 ref 1 00000100:00100000:2.0:1789762900.625810:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373959354949 time 373859354949 state 0x1 ref 1 00000100:00100000:0.0:1789762900.712380:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373959354949 time 373959446446 state 0x0 ref 1 00000100:00100000:0.0:1789762900.712402:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067494016:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762900.712448:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 74448896, mtime: 0x6aad9d54, atime: 0x0, ctime: 0x6aad9d54, blocks: 0x23809 00010000:00010000:0.0:1789762900.712455:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.712469:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=373s v=(0 0) 00010000:00010000:0.0:1789762900.712472:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762900.712477:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.712488:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762900.723779:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968002 is committed 00002000:00010000:2.0:1789762900.723790:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:2.0:1789762900.723795:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 145417 -> 147467 00010000:00010000:2.0:1789762900.723805:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.723814:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.723820:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.723825:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762900.723829:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.723842:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.723856:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762900.723861:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f176e33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762900.723913:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067494016:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11518us (98212us total) trans 4294968002 rc 0/0 00000100:00100000:2.0:1789762900.723947:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 795 00000100:00100000:2.0:1789762900.723955:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373959354949 time 373959446446 state 0x2 ref 0 00000100:00100000:2.0:1789762900.726053:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762900.726110:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067494144 00000100:00100000:1.0:1789762900.726118:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067494144 00000100:00100000:1.0:1789762900.726155:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 373959354949 time 373959446446 state 0x0 ref 1 00000100:00100000:1.0:1789762900.726166:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374059446446 time 373959446446 state 0x1 ref 1 00000100:00100000:0.0:1789762900.812465:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374059446446 time 374059532234 state 0x0 ref 1 00000100:00100000:0.0:1789762900.812487:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067494144:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762900.812541:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 75497472, mtime: 0x6aad9d54, atime: 0x0, ctime: 0x6aad9d54, blocks: 0x2400b 00010000:00010000:0.0:1789762900.812549:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176e3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.812562:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=374s v=(0 0) 00010000:00010000:0.0:1789762900.812566:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762900.812571:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176e3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.812584:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176e3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762900.828537:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968003 is committed 00002000:00010000:0.0:1789762900.828540:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 75497472 -> 76546048 00002000:00010000:0.0:1789762900.828546:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 147467 -> 149515 00010000:00010000:0.0:1789762900.828580:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176e3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.828600:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176e3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.828620:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176e3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.828636:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762900.828645:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176e3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.828665:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176e3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.828699:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176e3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762900.828715:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176e3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762900.828847:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067494144:12345-192.168.203.37@tcp:4:dd.500 Request processed in 16360us (102785us total) trans 4294968003 rc 0/0 00000100:00100000:0.0:1789762900.828864:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 796 00000100:00100000:0.0:1789762900.828880:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374059446446 time 374059532234 state 0x2 ref 0 00000100:00100000:3.0:1789762900.831377:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762900.831413:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067494272 00000100:00100000:0.0:1789762900.831422:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067494272 00000100:00100000:0.0:1789762900.831463:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374059446446 time 374059532234 state 0x0 ref 1 00000100:00100000:0.0:1789762900.831475:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374159532234 time 374059532234 state 0x1 ref 1 00000100:00100000:3.0:1789762900.912524:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374159532234 time 374159592471 state 0x0 ref 1 00000100:00100000:3.0:1789762900.912544:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067494272:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762900.912600:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 76546048, mtime: 0x6aad9d54, atime: 0x0, ctime: 0x6aad9d54, blocks: 0x2480b 00010000:00010000:3.0:1789762900.912607:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176e41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762900.912621:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=374s v=(0 0) 00010000:00010000:3.0:1789762900.912624:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762900.912629:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176e41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762900.912640:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176e41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762900.924469:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968004 is committed 00002000:00010000:1.0:1789762900.924510:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 76546048 -> 77594624 00002000:00010000:1.0:1789762900.924515:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 149515 -> 151563 00010000:00010000:1.0:1789762900.924524:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176e41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762900.924533:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176e41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762900.924540:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176e41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762900.924549:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762900.924553:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176e41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762900.924559:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176e41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762900.924584:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176e41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762900.924588:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176e41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762900.924646:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067494272:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12108us (93279us total) trans 4294968004 rc 0/0 00000100:00100000:1.0:1789762900.924653:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 797 00000100:00100000:1.0:1789762900.924662:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374159532234 time 374159592471 state 0x2 ref 0 00000100:00100000:1.0:1789762900.927044:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762900.927106:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067494400 00000100:00100000:2.0:1789762900.927115:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067494400 00000100:00100000:2.0:1789762900.927153:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374159532234 time 374159592471 state 0x0 ref 1 00000100:00100000:2.0:1789762900.927163:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374259592471 time 374159592471 state 0x1 ref 1 00000100:00100000:2.0:1789762901.012607:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374259592471 time 374259672083 state 0x0 ref 1 00000100:00100000:2.0:1789762901.012645:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067494400:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:0.0:1789762901.012668:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762901.012707:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 77594624, mtime: 0x6aad9d54, atime: 0x0, ctime: 0x6aad9d54, blocks: 0x2500b 00010000:00010000:2.0:1789762901.012715:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.012728:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=374s v=(0 0) 00010000:00010000:2.0:1789762901.012732:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762901.012737:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.012759:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762901.024139:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968005 is committed 00002000:00010000:1.0:1789762901.024163:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:1.0:1789762901.024169:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 151563 -> 153611 00010000:00010000:1.0:1789762901.024178:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.024186:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.024194:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.024200:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762901.024203:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.024209:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.024221:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.024226:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762901.024283:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067494400:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11659us (97238us total) trans 4294968005 rc 0/0 00000100:00100000:1.0:1789762901.024291:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 798 00000100:00100000:1.0:1789762901.024300:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374259592471 time 374259672083 state 0x2 ref 0 00000100:00100000:1.0:1789762901.026582:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762901.026636:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067494528 00000100:00100000:0.0:1789762901.026645:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067494528 00000100:00100000:0.0:1789762901.026684:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374259592471 time 374259672083 state 0x0 ref 1 00000100:00100000:0.0:1789762901.026695:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374359672083 time 374259672083 state 0x1 ref 1 00000100:00100000:2.0:1789762901.112673:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374359672083 time 374359739629 state 0x0 ref 1 00000100:00100000:2.0:1789762901.112692:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067494528:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762901.112738:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 78643200, mtime: 0x6aad9d54, atime: 0x0, ctime: 0x6aad9d54, blocks: 0x2580b 00010000:00010000:2.0:1789762901.112745:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176e4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.112759:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=374s v=(0 0) 00010000:00010000:2.0:1789762901.112762:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762901.112767:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176e4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.112781:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176e4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762901.123499:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968006 is committed 00002000:00010000:2.0:1789762901.123516:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:2.0:1789762901.123521:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 153611 -> 155659 00010000:00010000:2.0:1789762901.123532:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176e4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.123541:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176e4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.123547:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176e4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.123552:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762901.123556:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176e4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.123562:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176e4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.123577:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176e4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.123582:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176e4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762901.123636:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067494528:12345-192.168.203.37@tcp:4:dd.500 Request processed in 10948us (97053us total) trans 4294968006 rc 0/0 00000100:00100000:2.0:1789762901.123651:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 799 00000100:00100000:2.0:1789762901.123659:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374359672083 time 374359739629 state 0x2 ref 0 00000100:00100000:3.0:1789762901.126057:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762901.126143:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067494656 00000100:00100000:1.0:1789762901.126152:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067494656 00000100:00100000:1.0:1789762901.126204:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374359672083 time 374359739629 state 0x0 ref 1 00000100:00100000:1.0:1789762901.126220:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374459739629 time 374359739629 state 0x1 ref 1 00000100:00100000:1.0:1789762901.212738:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374459739629 time 374459803811 state 0x0 ref 1 00000100:00100000:1.0:1789762901.212760:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067494656:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762901.212805:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 79691776, mtime: 0x6aad9d54, atime: 0x0, ctime: 0x6aad9d54, blocks: 0x2600b 00010000:00010000:1.0:1789762901.212812:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176e56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.212824:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=374s v=(0 0) 00010000:00010000:1.0:1789762901.212828:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762901.212833:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176e56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.212844:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176e56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762901.224330:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 79691776 -> 80740352 00002000:00010000:2.0:1789762901.224336:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 155659 -> 157707 00010000:00010000:2.0:1789762901.224345:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176e56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.224355:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176e56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.224362:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176e56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762901.224365:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968007 is committed 00010000:00010000:2.0:1789762901.224368:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762d800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762901.224374:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176e56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.224382:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176e56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.224408:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176e56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.224413:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176e56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762901.224482:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067494656:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11729us (98424us total) trans 4294968007 rc 0/0 00000100:00100000:2.0:1789762901.224490:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 800 00000100:00100000:2.0:1789762901.224500:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374459739629 time 374459803811 state 0x2 ref 0 00000100:00100000:3.0:1789762901.226648:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762901.226692:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067494784 00000100:00100000:1.0:1789762901.226700:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067494784 00000100:00100000:1.0:1789762901.226738:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374459739629 time 374459803811 state 0x0 ref 1 00000100:00100000:1.0:1789762901.226748:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374559803811 time 374459803811 state 0x1 ref 1 00000100:00100000:1.0:1789762901.312796:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374559803811 time 374559860824 state 0x0 ref 1 00000100:00100000:1.0:1789762901.312819:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067494784:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762901.312865:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 80740352, mtime: 0x6aad9d54, atime: 0x0, ctime: 0x6aad9d54, blocks: 0x2680b 00010000:00010000:1.0:1789762901.312873:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176e5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.312886:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=374s v=(0 0) 00010000:00010000:1.0:1789762901.312890:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762901.312894:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176e5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.312906:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176e5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762901.324807:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 80740352 -> 81788928 00000001:00080000:0.0:1789762901.324808:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968008 is committed 00002000:00010000:2.0:1789762901.324813:0:8058:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb mtime from disk: 1789762900 -> 1789762901 00002000:00010000:2.0:1789762901.324817:0:8058:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb ctime from disk: 1789762900 -> 1789762901 00002000:00010000:2.0:1789762901.324820:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 157707 -> 159755 00010000:00010000:2.0:1789762901.324829:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176e5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.324839:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176e5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.324846:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176e5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.324851:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762901.324855:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176e5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.324861:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176e5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.324886:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176e5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.324892:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cabdf227/0xdf5a7ac59f176e5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762901.324974:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067494784:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12160us (98335us total) trans 4294968008 rc 0/0 00000100:00100000:2.0:1789762901.324981:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 801 00000100:00100000:2.0:1789762901.324990:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374559803811 time 374559860824 state 0x2 ref 0 00000100:00100000:3.0:1789762901.327240:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762901.327275:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067494912 00000100:00100000:2.0:1789762901.327283:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067494912 00000100:00100000:2.0:1789762901.327322:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374559803811 time 374559860824 state 0x0 ref 1 00000100:00100000:2.0:1789762901.327333:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374659860824 time 374559860824 state 0x1 ref 1 00000100:00100000:2.0:1789762901.412880:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374659860824 time 374659943656 state 0x0 ref 1 00000100:00100000:2.0:1789762901.412905:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067494912:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762901.412982:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 81788928, mtime: 0x6aad9d55, atime: 0x0, ctime: 0x6aad9d55, blocks: 0x2700b 00010000:00010000:2.0:1789762901.412990:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.413026:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=374s v=(0 0) 00010000:00010000:2.0:1789762901.413030:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762901.413035:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.413045:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762901.425188:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 81788928 -> 82837504 00000001:00080000:0.0:1789762901.425192:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968009 is committed 00002000:00010000:2.0:1789762901.425195:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 159755 -> 161803 00010000:00010000:2.0:1789762901.425208:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.425217:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.425224:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.425229:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762901.425233:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.425239:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.425264:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.425270:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176e64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762901.425337:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067494912:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12439us (98096us total) trans 4294968009 rc 0/0 00000100:00100000:2.0:1789762901.425347:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 802 00000100:00100000:2.0:1789762901.425356:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374659860824 time 374659943656 state 0x2 ref 0 00000100:00100000:2.0:1789762901.427706:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762901.427763:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067495040 00000100:00100000:1.0:1789762901.427772:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067495040 00000100:00100000:1.0:1789762901.427817:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374659860824 time 374659943656 state 0x0 ref 1 00000100:00100000:1.0:1789762901.427832:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374759943656 time 374659943656 state 0x1 ref 1 00000100:00100000:0.0:1789762901.512976:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374759943656 time 374760030488 state 0x0 ref 1 00000100:00100000:0.0:1789762901.513000:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067495040:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762901.513049:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 82837504, mtime: 0x6aad9d55, atime: 0x0, ctime: 0x6aad9d55, blocks: 0x2780b 00010000:00010000:0.0:1789762901.513057:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176e6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.513084:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=374s v=(0 0) 00010000:00010000:0.0:1789762901.513104:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762901.513112:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176e6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.513127:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176e6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762901.525011:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968010 is committed 00002000:00010000:1.0:1789762901.525020:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 82837504 -> 83886080 00002000:00010000:1.0:1789762901.525026:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 161803 -> 163851 00010000:00010000:1.0:1789762901.525036:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176e6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.525045:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176e6b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.525052:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176e6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.525058:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762901.525061:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176e6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.525068:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176e6b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.525082:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176e6b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.525105:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176e6b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762901.525178:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067495040:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12182us (97468us total) trans 4294968010 rc 0/0 00000100:00100000:1.0:1789762901.525189:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 803 00000100:00100000:1.0:1789762901.525201:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374759943656 time 374760030488 state 0x2 ref 0 00000100:00100000:1.0:1789762901.527525:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762901.527568:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067495168 00000100:00100000:2.0:1789762901.527576:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067495168 00000100:00100000:2.0:1789762901.527615:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374759943656 time 374760030488 state 0x0 ref 1 00000100:00100000:2.0:1789762901.527625:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374860030488 time 374760030488 state 0x1 ref 1 00000100:00100000:1.0:1789762901.613103:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374860030488 time 374860154696 state 0x0 ref 1 00000100:00100000:1.0:1789762901.613130:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067495168:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762901.613187:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 83886080, mtime: 0x6aad9d55, atime: 0x0, ctime: 0x6aad9d55, blocks: 0x2800b 00010000:00010000:1.0:1789762901.613198:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176e72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.613212:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=374s v=(0 0) 00010000:00010000:1.0:1789762901.613216:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762901.613220:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176e72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.613232:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176e72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762901.624436:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968011 is committed 00002000:00010000:0.0:1789762901.624457:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 83886080 -> 84934656 00002000:00010000:0.0:1789762901.624463:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 163851 -> 165899 00010000:00010000:0.0:1789762901.624473:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176e72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.624483:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176e72 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.624490:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176e72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.624496:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762d200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762901.624499:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176e72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.624505:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176e72 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.624523:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176e72 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.624529:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007cfd0e94/0xdf5a7ac59f176e72 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762901.624598:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067495168:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11477us (97071us total) trans 4294968011 rc 0/0 00000100:00100000:0.0:1789762901.624605:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 804 00000100:00100000:0.0:1789762901.624614:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374860030488 time 374860154696 state 0x2 ref 0 00000100:00100000:2.0:1789762901.626909:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762901.626953:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067495296 00000100:00100000:3.0:1789762901.626961:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067495296 00000100:00100000:3.0:1789762901.626999:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374860030488 time 374860154696 state 0x0 ref 1 00000100:00100000:3.0:1789762901.627010:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374960154696 time 374860154696 state 0x1 ref 1 00010000:00010000:0.0:1789762901.632025:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762901.713207:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374960154696 time 374960261554 state 0x0 ref 1 00000100:00100000:1.0:1789762901.713232:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067495296:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762901.713281:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 84934656, mtime: 0x6aad9d55, atime: 0x0, ctime: 0x6aad9d55, blocks: 0x2880b 00010000:00010000:1.0:1789762901.713288:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176e79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.713302:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=374s v=(0 0) 00010000:00010000:1.0:1789762901.713306:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762901.713310:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176e79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.713322:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176e79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762901.725070:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 84934656 -> 85983232 00002000:00010000:2.0:1789762901.725076:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 165899 -> 167947 00000001:00080000:3.0:1789762901.725080:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968012 is committed 00010000:00010000:2.0:1789762901.725099:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176e79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.725127:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176e79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.725138:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176e79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.725146:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762901.725150:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176e79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.725157:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176e79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.725173:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176e79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.725178:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000141af078/0xdf5a7ac59f176e79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762901.725237:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067495296:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12014us (98327us total) trans 4294968012 rc 0/0 00000100:00100000:2.0:1789762901.725245:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 805 00000100:00100000:2.0:1789762901.725253:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374960154696 time 374960261554 state 0x2 ref 0 00000100:00100000:0.0:1789762901.727575:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762901.727628:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067495424 00000100:00100000:1.0:1789762901.727637:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067495424 00000100:00100000:1.0:1789762901.727679:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 374960154696 time 374960261554 state 0x0 ref 1 00000100:00100000:1.0:1789762901.727690:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375060261554 time 374960261554 state 0x1 ref 1 00000100:00100000:1.0:1789762901.813252:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375060261554 time 375060316503 state 0x0 ref 1 00000100:00100000:1.0:1789762901.813276:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067495424:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762901.813326:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 85983232, mtime: 0x6aad9d55, atime: 0x0, ctime: 0x6aad9d55, blocks: 0x2900b 00010000:00010000:1.0:1789762901.813333:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176e80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.813347:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=375s v=(0 0) 00010000:00010000:1.0:1789762901.813350:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762901.813355:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176e80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762901.813368:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176e80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762901.824329:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968013 is committed 00002000:00010000:0.0:1789762901.824391:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 85983232 -> 87031808 00002000:00010000:0.0:1789762901.824396:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 167947 -> 169995 00010000:00010000:0.0:1789762901.824405:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176e80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.824414:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176e80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.824420:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176e80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.824426:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762dc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762901.824430:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176e80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.824436:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176e80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.824450:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176e80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762901.824455:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176e80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762901.824514:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067495424:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11246us (96938us total) trans 4294968013 rc 0/0 00000100:00100000:0.0:1789762901.824523:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 806 00000100:00100000:0.0:1789762901.824532:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375060261554 time 375060316503 state 0x2 ref 0 00000100:00100000:0.0:1789762901.826901:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762901.826987:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067495552 00000100:00100000:1.0:1789762901.826995:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067495552 00000100:00100000:1.0:1789762901.827043:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375060261554 time 375060316503 state 0x0 ref 1 00000100:00100000:1.0:1789762901.827060:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375160316503 time 375060316503 state 0x1 ref 1 00000100:00100000:2.0:1789762901.913351:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375160316503 time 375160412551 state 0x0 ref 1 00000100:00100000:2.0:1789762901.913401:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067495552:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762901.913457:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 87031808, mtime: 0x6aad9d55, atime: 0x0, ctime: 0x6aad9d55, blocks: 0x2980b 00010000:00010000:2.0:1789762901.913466:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e87 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.913482:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=375s v=(0 0) 00010000:00010000:2.0:1789762901.913487:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762901.913494:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.913509:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762901.925682:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 87031808 -> 88080384 00002000:00010000:2.0:1789762901.925688:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 169995 -> 172043 00000001:00080000:3.0:1789762901.925688:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968014 is committed 00010000:00010000:2.0:1789762901.925698:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.925708:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.925716:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.925721:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762901.925731:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.925739:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.925759:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762901.925768:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762901.925833:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067495552:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12450us (98931us total) trans 4294968014 rc 0/0 00000100:00100000:2.0:1789762901.925841:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 807 00000100:00100000:2.0:1789762901.925850:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375160316503 time 375160412551 state 0x2 ref 0 00000100:00100000:2.0:1789762901.928223:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762901.928264:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067495680 00000100:00100000:1.0:1789762901.928273:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067495680 00000100:00100000:1.0:1789762901.928309:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375160316503 time 375160412551 state 0x0 ref 1 00000100:00100000:1.0:1789762901.928320:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375260412551 time 375160412551 state 0x1 ref 1 00000100:00100000:2.0:1789762902.013444:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375260412551 time 375260510308 state 0x0 ref 1 00000100:00100000:2.0:1789762902.013466:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067495680:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762902.013517:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 88080384, mtime: 0x6aad9d55, atime: 0x0, ctime: 0x6aad9d55, blocks: 0x2a00b 00010000:00010000:2.0:1789762902.013524:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.013537:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=375s v=(0 0) 00010000:00010000:2.0:1789762902.013551:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762902.013556:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.013567:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762902.024950:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968015 is committed 00002000:00010000:0.0:1789762902.024956:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 88080384 -> 89128960 00002000:00010000:0.0:1789762902.024962:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 172043 -> 174091 00010000:00010000:0.0:1789762902.024973:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.024983:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.024991:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.024998:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762902.025002:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.025008:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.025029:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.025034:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176e8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762902.025115:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067495680:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11654us (96890us total) trans 4294968015 rc 0/0 00000100:00100000:0.0:1789762902.025124:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 808 00000100:00100000:0.0:1789762902.025133:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375260412551 time 375260510308 state 0x2 ref 0 00000100:00100000:2.0:1789762902.027477:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762902.027515:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067495808 00000100:00100000:0.0:1789762902.027522:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067495808 00000100:00100000:0.0:1789762902.027559:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375260412551 time 375260510308 state 0x0 ref 1 00000100:00100000:0.0:1789762902.027570:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375360510308 time 375260510308 state 0x1 ref 1 00000100:00100000:3.0:1789762902.113535:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375360510308 time 375360598923 state 0x0 ref 1 00000100:00100000:3.0:1789762902.113560:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067495808:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762902.113603:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 89128960, mtime: 0x6aad9d55, atime: 0x0, ctime: 0x6aad9d55, blocks: 0x2a80b 00010000:00010000:3.0:1789762902.113610:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f176e95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762902.113623:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=375s v=(0 0) 00010000:00010000:3.0:1789762902.113627:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762902.113632:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f176e95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762902.113642:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f176e95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762902.124995:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968016 is committed 00002000:00010000:2.0:1789762902.124998:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 89128960 -> 90177536 00002000:00010000:2.0:1789762902.125003:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 174091 -> 176139 00010000:00010000:2.0:1789762902.125012:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f176e95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.125021:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f176e95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.125028:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f176e95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.125033:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762902.125037:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f176e95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.125044:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f176e95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.125057:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f176e95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.125068:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f176e95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762902.125133:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067495808:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11580us (97656us total) trans 4294968016 rc 0/0 00000100:00100000:2.0:1789762902.125140:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 809 00000100:00100000:2.0:1789762902.125148:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375360510308 time 375360598923 state 0x2 ref 0 00000100:00100000:1.0:1789762902.127434:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762902.127500:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067495936 00000100:00100000:2.0:1789762902.127509:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067495936 00000100:00100000:2.0:1789762902.127544:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375360510308 time 375360598923 state 0x0 ref 1 00000100:00100000:2.0:1789762902.127555:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375460598923 time 375360598923 state 0x1 ref 1 00000100:00100000:0.0:1789762902.213646:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375460598923 time 375460704910 state 0x0 ref 1 00000100:00100000:0.0:1789762902.213670:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067495936:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762902.213720:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 90177536, mtime: 0x6aad9d55, atime: 0x0, ctime: 0x6aad9d55, blocks: 0x2b00b 00010000:00010000:0.0:1789762902.213728:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f176e9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.213740:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=375s v=(0 0) 00010000:00010000:0.0:1789762902.213744:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762902.213749:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f176e9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.213760:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f176e9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762902.224599:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968017 is committed 00002000:00010000:0.0:1789762902.224620:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 90177536 -> 91226112 00002000:00010000:0.0:1789762902.224627:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 176139 -> 178187 00010000:00010000:0.0:1789762902.224638:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f176e9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.224651:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f176e9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.224661:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f176e9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.224669:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762902.224675:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f176e9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.224684:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f176e9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.224700:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f176e9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.224705:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f176e9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762902.224759:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067495936:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11095us (97323us total) trans 4294968017 rc 0/0 00000100:00100000:0.0:1789762902.224769:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 810 00000100:00100000:0.0:1789762902.224779:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375460598923 time 375460704910 state 0x2 ref 0 00000100:00100000:0.0:1789762902.226894:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762902.226969:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067496064 00000100:00100000:1.0:1789762902.226977:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067496064 00000100:00100000:1.0:1789762902.227015:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375460598923 time 375460704910 state 0x0 ref 1 00000100:00100000:1.0:1789762902.227026:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375560704910 time 375460704910 state 0x1 ref 1 00000100:00100000:1.0:1789762902.313722:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375560704910 time 375560778925 state 0x0 ref 1 00000100:00100000:1.0:1789762902.313744:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067496064:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762902.313791:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 91226112, mtime: 0x6aad9d55, atime: 0x0, ctime: 0x6aad9d55, blocks: 0x2b80b 00010000:00010000:1.0:1789762902.313798:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ea3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762902.313815:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=375s v=(0 0) 00010000:00010000:1.0:1789762902.313818:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762902.313823:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ea3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762902.313833:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ea3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762902.326264:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968018 is committed 00002000:00010000:1.0:1789762902.326265:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 91226112 -> 92274688 00002000:00010000:1.0:1789762902.326271:0:8059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb mtime from disk: 1789762901 -> 1789762902 00002000:00010000:1.0:1789762902.326274:0:8059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb ctime from disk: 1789762901 -> 1789762902 00002000:00010000:1.0:1789762902.326276:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 178187 -> 180235 00010000:00010000:1.0:1789762902.326285:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ea3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762902.326295:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ea3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762902.326301:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ea3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762902.326306:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762dc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762902.326309:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ea3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762902.326316:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ea3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762902.326342:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ea3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762902.326347:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176ea3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762902.326416:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067496064:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12677us (99521us total) trans 4294968018 rc 0/0 00000100:00100000:1.0:1789762902.326424:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 811 00000100:00100000:1.0:1789762902.326433:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375560704910 time 375560778925 state 0x2 ref 0 00000100:00100000:2.0:1789762902.328930:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762902.328982:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067496192 00000100:00100000:3.0:1789762902.328990:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067496192 00000100:00100000:3.0:1789762902.329037:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375560704910 time 375560778925 state 0x0 ref 1 00000100:00100000:3.0:1789762902.329053:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375660778925 time 375560778925 state 0x1 ref 1 00000100:00100000:1.0:1789762902.413818:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375660778925 time 375660883490 state 0x0 ref 1 00000100:00100000:1.0:1789762902.413842:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff70dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067496192:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762902.413890:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 92274688, mtime: 0x6aad9d56, atime: 0x0, ctime: 0x6aad9d56, blocks: 0x2c00b 00010000:00010000:1.0:1789762902.413897:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176eaa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762902.413910:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=375s v=(0 0) 00010000:00010000:1.0:1789762902.413914:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762902.413937:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176eaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762902.413949:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176eaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762902.425265:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968019 is committed 00002000:00010000:2.0:1789762902.425266:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 92274688 -> 93323264 00002000:00010000:2.0:1789762902.425273:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 180235 -> 182283 00010000:00010000:2.0:1789762902.425283:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176eaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.425292:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176eaa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.425299:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176eaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.425311:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762dc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762902.425315:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176eaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.425322:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176eaa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.425336:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176eaa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.425341:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c8f0e49/0xdf5a7ac59f176eaa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762902.425398:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff70dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067496192:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11564us (96481us total) trans 4294968019 rc 0/0 00000100:00100000:2.0:1789762902.425406:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 812 00000100:00100000:2.0:1789762902.425415:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375660778925 time 375660883490 state 0x2 ref 0 00000100:00100000:0.0:1789762902.427701:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762902.427770:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067496320 00000100:00100000:2.0:1789762902.427778:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067496320 00000100:00100000:2.0:1789762902.427813:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375660778925 time 375660883490 state 0x0 ref 1 00000100:00100000:2.0:1789762902.427824:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375760883490 time 375660883490 state 0x1 ref 1 00000100:00100000:0.0:1789762902.513908:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375760883490 time 375760973771 state 0x0 ref 1 00000100:00100000:0.0:1789762902.513947:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067496320:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762902.513995:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 93323264, mtime: 0x6aad9d56, atime: 0x0, ctime: 0x6aad9d56, blocks: 0x2c80b 00010000:00010000:0.0:1789762902.514002:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f176eb1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.514020:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=375s v=(0 0) 00010000:00010000:0.0:1789762902.514024:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762902.514029:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f176eb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.514040:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f176eb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762902.526501:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 93323264 -> 94371840 00000001:00080000:0.0:1789762902.526510:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968020 is committed 00002000:00010000:2.0:1789762902.526513:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 182283 -> 184331 00010000:00010000:2.0:1789762902.526525:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f176eb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.526535:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f176eb1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.526542:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f176eb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.526548:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762902.526551:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f176eb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.526558:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f176eb1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.526573:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f176eb1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.526578:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f176eb1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762902.526638:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067496320:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12698us (98936us total) trans 4294968020 rc 0/0 00000100:00100000:2.0:1789762902.526646:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 813 00000100:00100000:2.0:1789762902.526655:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375760883490 time 375760973771 state 0x2 ref 0 00000100:00100000:0.0:1789762902.528968:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762902.528999:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067496448 00000100:00100000:3.0:1789762902.529007:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067496448 00000100:00100000:0.0:1789762902.529024:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37eb90c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074724992:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762902.529049:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375760883490 time 375760973771 state 0x0 ref 1 00000100:00100000:0.0:1789762902.529065:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762902.529069:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375860973771 time 375760973771 state 0x1 ref 1 00000100:00100000:1.0:1789762902.529117:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074724992 00000100:00100000:1.0:1789762902.529123:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074724992 00000100:00100000:1.0:1789762902.529178:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:1.0:1789762902.529191:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074724992:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762902.529224:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762902.529277:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074724992:12345-0@lo:41:osp-pre-1-0.0 Request processed in 89us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762902.529287:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00100000:0.0:1789762902.529321:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37eb90c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074724992:0@lo:41:osp-pre-1-0.0 00000001:00080000:0.0:1789762902.537669:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967417 is committed 00000100:00100000:2.0:1789762902.614017:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375860973771 time 375861082933 state 0x0 ref 1 00000100:00100000:2.0:1789762902.614040:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067496448:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762902.614106:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 94371840, mtime: 0x6aad9d56, atime: 0x0, ctime: 0x6aad9d56, blocks: 0x2d00b 00010000:00010000:2.0:1789762902.614116:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176eb8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.614134:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=375s v=(0 0) 00010000:00010000:2.0:1789762902.614139:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762902.614146:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176eb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.614161:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176eb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762902.626690:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 94371840 -> 95420416 00002000:00010000:2.0:1789762902.626696:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 184331 -> 186379 00000001:00080000:3.0:1789762902.626700:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968021 is committed 00010000:00010000:2.0:1789762902.626706:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176eb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.626717:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176eb8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.626724:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176eb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.626740:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762902.626743:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176eb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.626749:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176eb8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.626764:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176eb8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762902.626769:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f176eb8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762902.626827:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067496448:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12793us (97861us total) trans 4294968021 rc 0/0 00000100:00100000:2.0:1789762902.626836:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 814 00000100:00100000:2.0:1789762902.626844:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375860973771 time 375861082933 state 0x2 ref 0 00000100:00100000:1.0:1789762902.629005:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762902.629039:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067496576 00000100:00100000:2.0:1789762902.629047:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067496576 00000100:00100000:2.0:1789762902.629098:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375860973771 time 375861082933 state 0x0 ref 1 00000100:00100000:2.0:1789762902.629110:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375961082933 time 375861082933 state 0x1 ref 1 00010000:00010000:1.0:1789762902.656062:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762902.714119:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375961082933 time 375961184640 state 0x0 ref 1 00000100:00100000:1.0:1789762902.714142:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067496576:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762902.714188:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 95420416, mtime: 0x6aad9d56, atime: 0x0, ctime: 0x6aad9d56, blocks: 0x2d80b 00010000:00010000:1.0:1789762902.714196:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176ebf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762902.714210:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=375s v=(0 0) 00010000:00010000:1.0:1789762902.714215:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762902.714222:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176ebf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762902.714236:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176ebf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762902.725544:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968022 is committed 00002000:00010000:0.0:1789762902.725598:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 95420416 -> 96468992 00002000:00010000:0.0:1789762902.725603:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 186379 -> 188427 00010000:00010000:0.0:1789762902.725612:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176ebf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.725621:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176ebf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.725627:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176ebf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.725636:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762d400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762902.725640:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176ebf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.725645:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176ebf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.725658:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176ebf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.725663:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ef1e2fa/0xdf5a7ac59f176ebf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762902.725722:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067496576:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11588us (96717us total) trans 4294968022 rc 0/0 00000100:00100000:0.0:1789762902.725730:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 815 00000100:00100000:0.0:1789762902.725738:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375961082933 time 375961184640 state 0x2 ref 0 00000100:00100000:0.0:1789762902.727932:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762902.728004:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067496704 00000100:00100000:2.0:1789762902.728018:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067496704 00000100:00100000:2.0:1789762902.728055:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 375961082933 time 375961184640 state 0x0 ref 1 00000100:00100000:2.0:1789762902.728065:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376061184640 time 375961184640 state 0x1 ref 1 00000100:00100000:3.0:1789762902.814199:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376061184640 time 376061265416 state 0x0 ref 1 00000100:00100000:3.0:1789762902.814219:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067496704:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762902.814265:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 96468992, mtime: 0x6aad9d56, atime: 0x0, ctime: 0x6aad9d56, blocks: 0x2e00b 00010000:00010000:3.0:1789762902.814272:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ec6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762902.814293:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=376s v=(0 0) 00010000:00010000:3.0:1789762902.814297:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762902.814302:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ec6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762902.814316:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ec6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762902.825481:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968023 is committed 00002000:00010000:3.0:1789762902.825522:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 96468992 -> 97517568 00002000:00010000:3.0:1789762902.825527:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 188427 -> 190475 00010000:00010000:3.0:1789762902.825537:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ec6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762902.825547:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ec6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762902.825553:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ec6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762902.825558:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762902.825561:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ec6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762902.825567:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ec6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762902.825580:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ec6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762902.825586:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ec6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762902.825647:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067496704:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11430us (97727us total) trans 4294968023 rc 0/0 00000100:00100000:3.0:1789762902.825658:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 816 00000100:00100000:3.0:1789762902.825670:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376061184640 time 376061265416 state 0x2 ref 0 00000100:00100000:1.0:1789762902.827938:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762902.827982:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067496832 00000100:00100000:2.0:1789762902.827990:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067496832 00000100:00100000:2.0:1789762902.828025:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376061184640 time 376061265416 state 0x0 ref 1 00000100:00100000:2.0:1789762902.828036:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376161265416 time 376061265416 state 0x1 ref 1 00000100:00100000:0.0:1789762902.914294:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376161265416 time 376161359568 state 0x0 ref 1 00000100:00100000:0.0:1789762902.914317:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067496832:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762902.914366:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 97517568, mtime: 0x6aad9d56, atime: 0x0, ctime: 0x6aad9d56, blocks: 0x2e80b 00010000:00010000:0.0:1789762902.914375:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176ecd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.914388:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=376s v=(0 0) 00010000:00010000:0.0:1789762902.914391:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762902.914405:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176ecd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.914416:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176ecd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762902.926071:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968024 is committed 00002000:00010000:0.0:1789762902.926098:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 97517568 -> 98566144 00002000:00010000:0.0:1789762902.926103:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 190475 -> 192523 00010000:00010000:0.0:1789762902.926112:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176ecd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.926121:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176ecd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.926128:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176ecd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.926133:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762902.926136:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176ecd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.926143:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176ecd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.926158:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176ecd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762902.926163:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176ecd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762902.926217:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067496832:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11906us (98295us total) trans 4294968024 rc 0/0 00000100:00100000:0.0:1789762902.926225:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 817 00000100:00100000:0.0:1789762902.926235:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376161265416 time 376161359568 state 0x2 ref 0 00000100:00100000:2.0:1789762902.928339:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762902.928370:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067496960 00000100:00100000:0.0:1789762902.928378:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067496960 00000100:00100000:0.0:1789762902.928422:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376161265416 time 376161359568 state 0x0 ref 1 00000100:00100000:0.0:1789762902.928433:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376261359568 time 376161359568 state 0x1 ref 1 00000100:00100000:3.0:1789762903.014378:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376261359568 time 376261445213 state 0x0 ref 1 00000100:00100000:3.0:1789762903.014412:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067496960:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:0.0:1789762903.014435:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789762903.014456:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 98566144, mtime: 0x6aad9d56, atime: 0x0, ctime: 0x6aad9d56, blocks: 0x2f00b 00010000:00010000:3.0:1789762903.014464:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ed4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.014480:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=376s v=(0 0) 00010000:00010000:3.0:1789762903.014485:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762903.014491:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ed4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.014507:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ed4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762903.025517:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 98566144 -> 99614720 00000001:00080000:0.0:1789762903.025521:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968025 is committed 00002000:00010000:1.0:1789762903.025525:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 192523 -> 194571 00010000:00010000:1.0:1789762903.025539:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ed4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.025554:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ed4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.025564:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ed4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.025572:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762903.025578:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ed4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.025589:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ed4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.025607:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ed4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.025613:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000c687ca/0xdf5a7ac59f176ed4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762903.025673:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067496960:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11279us (97332us total) trans 4294968025 rc 0/0 00000100:00100000:1.0:1789762903.025681:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 818 00000100:00100000:1.0:1789762903.025689:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376261359568 time 376261445213 state 0x2 ref 0 00000100:00100000:1.0:1789762903.028674:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762903.028712:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067497088 00000100:00100000:3.0:1789762903.028723:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067497088 00000100:00100000:3.0:1789762903.028782:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376261359568 time 376261445213 state 0x0 ref 1 00000100:00100000:3.0:1789762903.028795:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376361445213 time 376261445213 state 0x1 ref 1 00000100:00100000:1.0:1789762903.114491:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376361445213 time 376361556250 state 0x0 ref 1 00000100:00100000:1.0:1789762903.114512:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067497088:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762903.114559:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 99614720, mtime: 0x6aad9d56, atime: 0x0, ctime: 0x6aad9d56, blocks: 0x2f80b 00010000:00010000:1.0:1789762903.114566:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176edb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.114579:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=376s v=(0 0) 00010000:00010000:1.0:1789762903.114583:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762903.114587:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176edb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.114603:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176edb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762903.126355:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968026 is committed 00002000:00010000:3.0:1789762903.126408:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 99614720 -> 100663296 00002000:00010000:3.0:1789762903.126413:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 194571 -> 196619 00010000:00010000:3.0:1789762903.126422:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176edb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.126435:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176edb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.126441:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176edb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.126446:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762d800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762903.126449:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176edb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.126455:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176edb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.126469:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176edb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.126475:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000534df100/0xdf5a7ac59f176edb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762903.126532:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067497088:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12026us (97863us total) trans 4294968026 rc 0/0 00000100:00100000:3.0:1789762903.126540:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 819 00000100:00100000:3.0:1789762903.126549:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376361445213 time 376361556250 state 0x2 ref 0 00000100:00100000:2.0:1789762903.128887:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762903.128967:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067497216 00000100:00100000:0.0:1789762903.128986:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067497216 00000100:00100000:0.0:1789762903.129023:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376361445213 time 376361556250 state 0x0 ref 1 00000100:00100000:0.0:1789762903.129034:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376461556250 time 376361556250 state 0x1 ref 1 00000100:00100000:2.0:1789762903.214618:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376461556250 time 376461684733 state 0x0 ref 1 00000100:00100000:2.0:1789762903.214640:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067497216:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762903.214698:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 100663296, mtime: 0x6aad9d56, atime: 0x0, ctime: 0x6aad9d56, blocks: 0x3000b 00010000:00010000:2.0:1789762903.214707:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176ee2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762903.214723:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=376s v=(0 0) 00010000:00010000:2.0:1789762903.214727:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762903.214731:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176ee2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762903.214743:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176ee2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762903.226506:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968027 is committed 00002000:00010000:0.0:1789762903.226507:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 100663296 -> 101711872 00002000:00010000:0.0:1789762903.226514:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 196619 -> 198667 00010000:00010000:0.0:1789762903.226524:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176ee2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.226533:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176ee2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.226540:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176ee2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.226546:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762903.226549:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176ee2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.226556:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176ee2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.226570:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176ee2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.226575:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000965e748b/0xdf5a7ac59f176ee2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762903.226631:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067497216:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11998us (97744us total) trans 4294968027 rc 0/0 00000100:00100000:0.0:1789762903.226642:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 820 00000100:00100000:0.0:1789762903.226651:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376461556250 time 376461684733 state 0x2 ref 0 00000100:00100000:1.0:1789762903.229157:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762903.229201:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067497344 00000100:00100000:3.0:1789762903.229209:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067497344 00000100:00100000:3.0:1789762903.229248:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376461556250 time 376461684733 state 0x0 ref 1 00000100:00100000:3.0:1789762903.229259:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376561684733 time 376461684733 state 0x1 ref 1 00000100:00100000:2.0:1789762903.314699:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376561684733 time 376561764960 state 0x0 ref 1 00000100:00100000:2.0:1789762903.314720:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067497344:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762903.314766:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 101711872, mtime: 0x6aad9d56, atime: 0x0, ctime: 0x6aad9d56, blocks: 0x3080b 00010000:00010000:2.0:1789762903.314774:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176ee9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762903.314787:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=376s v=(0 0) 00010000:00010000:2.0:1789762903.314791:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762903.314795:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176ee9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762903.314805:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176ee9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762903.327320:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968028 is committed 00002000:00010000:0.0:1789762903.327342:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 101711872 -> 102760448 00002000:00010000:0.0:1789762903.327348:0:8057:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb mtime from disk: 1789762902 -> 1789762903 00002000:00010000:0.0:1789762903.327350:0:8057:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb ctime from disk: 1789762902 -> 1789762903 00002000:00010000:0.0:1789762903.327352:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 198667 -> 200715 00010000:00010000:0.0:1789762903.327361:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176ee9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.327370:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176ee9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.327377:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176ee9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.327382:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5861400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762903.327386:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176ee9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.327392:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176ee9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.327406:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176ee9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.327416:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005212985a/0xdf5a7ac59f176ee9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762903.327473:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067497344:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12758us (98315us total) trans 4294968028 rc 0/0 00000100:00100000:0.0:1789762903.327481:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 821 00000100:00100000:0.0:1789762903.327490:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376561684733 time 376561764960 state 0x2 ref 0 00000100:00100000:2.0:1789762903.330127:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762903.330180:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067497472 00000100:00100000:3.0:1789762903.330187:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067497472 00000100:00100000:3.0:1789762903.330223:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376561684733 time 376561764960 state 0x0 ref 1 00000100:00100000:3.0:1789762903.330234:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376661764960 time 376561764960 state 0x1 ref 1 00000100:00100000:1.0:1789762903.414780:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376661764960 time 376661845345 state 0x0 ref 1 00000100:00100000:1.0:1789762903.414812:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067497472:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762903.414857:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 102760448, mtime: 0x6aad9d57, atime: 0x0, ctime: 0x6aad9d57, blocks: 0x3100b 00010000:00010000:1.0:1789762903.414865:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176ef0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.414878:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=376s v=(0 0) 00010000:00010000:1.0:1789762903.414882:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762903.414886:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176ef0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.414897:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176ef0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762903.426438:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968029 is committed 00002000:00010000:0.0:1789762903.426450:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 102760448 -> 103809024 00002000:00010000:0.0:1789762903.426456:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 200715 -> 202763 00010000:00010000:0.0:1789762903.426466:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176ef0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.426476:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176ef0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.426482:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176ef0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.426487:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762c400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762903.426490:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176ef0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.426498:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176ef0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.426512:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176ef0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.426518:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ebb717/0xdf5a7ac59f176ef0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762903.426574:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067497472:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11777us (96445us total) trans 4294968029 rc 0/0 00000100:00100000:0.0:1789762903.426585:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 822 00000100:00100000:0.0:1789762903.426596:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376661764960 time 376661845345 state 0x2 ref 0 00000100:00100000:2.0:1789762903.428744:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762903.428782:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067497600 00000100:00100000:0.0:1789762903.428791:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067497600 00000100:00100000:0.0:1789762903.428829:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376661764960 time 376661845345 state 0x0 ref 1 00000100:00100000:0.0:1789762903.428840:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376761845345 time 376661845345 state 0x1 ref 1 00000100:00100000:0.0:1789762903.514842:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376761845345 time 376761907751 state 0x0 ref 1 00000100:00100000:0.0:1789762903.514863:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9af100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067497600:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762903.514954:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 103809024, mtime: 0x6aad9d57, atime: 0x0, ctime: 0x6aad9d57, blocks: 0x3180b 00010000:00010000:0.0:1789762903.514963:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f176ef7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.514981:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=376s v=(0 0) 00010000:00010000:0.0:1789762903.514985:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.514989:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f176ef7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.515000:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f176ef7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762903.526240:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968030 is committed 00002000:00010000:3.0:1789762903.526296:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb size from disk: 103809024 -> 104857600 00002000:00010000:3.0:1789762903.526301:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x8:0x0] updating lvb blocks from disk: 202763 -> 204811 00010000:00010000:3.0:1789762903.526311:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f176ef7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.526321:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f176ef7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.526327:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f176ef7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.526332:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762903.526337:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f176ef7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.526348:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f176ef7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.526361:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f176ef7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.526366:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f176ef7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762903.526423:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9af100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067497600:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11566us (97679us total) trans 4294968030 rc 0/0 00000100:00100000:3.0:1789762903.526431:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 823 00000100:00100000:3.0:1789762903.526439:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376761845345 time 376761907751 state 0x2 ref 0 00000100:00100000:2.0:1789762903.527771:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762903.527806:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067497856 00000100:00100000:3.0:1789762903.527813:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067497856 00000100:00100000:3.0:1789762903.527857:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 96 00000100:00100000:3.0:1789762903.527866:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15197:x1876702067497856:12345-192.168.203.37@tcp:35:dd.500 00000100:00080000:0.0:1789762903.527868:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789762903.528090:0:13186:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967435 00000100:00100000:3.0:1789762903.528185:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:15197:x1876702067497856:12345-192.168.203.37@tcp:35:dd.500 Request processed in 321us (414us total) trans 4294967436 rc 0/0 00000100:00100000:3.0:1789762903.528191:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 96 00000020:00080000:3.0:1789762903.528194:0:13186:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1789762903.552058:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074725120:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762903.552133:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762903.552174:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074725120 00000100:00100000:2.0:1789762903.552181:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074725120 00000100:00100000:2.0:1789762903.552224:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:2.0:1789762903.552232:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074725120:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762903.552260:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:2.0:1789762903.552327:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074725120:12345-0@lo:41:osp-pre-0-1.0 Request processed in 96us (195us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762903.552345:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00100000:3.0:1789762903.552378:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074725120:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762903.570452:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762903.570499:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067498112 00000100:00100000:3.0:1789762903.570507:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067498112 00000100:00100000:3.0:1789762903.570554:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 331 00000100:00100000:3.0:1789762903.570562:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:15214:x1876702067498112:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762903.570581:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762903.570597:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6e00/0xdf5a7ac59f176efe lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca5b expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.570688:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7400/0xdf5a7ac59f176f05 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.570700:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7400/0xdf5a7ac59f176f05 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.570711:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7400/0xdf5a7ac59f176f05 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762903.570723:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1c:0x0]: returning size 104857600 00010000:00010000:3.0:1789762903.570836:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af6e00/0xdf5a7ac59f176efe lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca5b expref: 13 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.570850:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7400/0xdf5a7ac59f176f05 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca5b expref: 12 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.570856:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff9c37e37a7e00) 00010000:00010000:3.0:1789762903.570859:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:3.0:1789762903.570862:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36c9af7400, rc 0) 00000100:00100000:3.0:1789762903.570905:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:15214:x1876702067498112:12345-192.168.203.37@tcp:101:dd.500 Request processed in 343us (459us total) trans 4294967437 rc 0/0 00000100:00100000:3.0:1789762903.570910:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 331 00000100:00100000:1.0:1789762903.572216:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762903.572241:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067498240 00000100:00100000:3.0:1789762903.572247:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067498240 00000100:00100000:3.0:1789762903.572278:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 200 00000100:00100000:3.0:1789762903.572284:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb90c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067498240:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762903.572293:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762903.572304:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176f13 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca62 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762903.572321:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 104857600, mtime: 0x6aad9d57, atime: 0x0, ctime: 0x6aad9d57, blocks: 0x3200b 00010000:00010000:3.0:1789762903.572333:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=376s v=(0 0) 00010000:00010000:3.0:1789762903.572336:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762903.572343:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176f13 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca62 expref: 6 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.572354:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176f13 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca62 expref: 6 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.572362:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36c9af6800, rc 0) 00000100:00100000:3.0:1789762903.572387:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb90c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067498240:12345-192.168.203.37@tcp:101:dd.500 Request processed in 103us (173us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762903.572391:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 200 00000100:00100000:2.0:1789762903.574480:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762903.574561:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067498368 00000100:00100000:0.0:1789762903.574567:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067498368 00000100:00100000:0.0:1789762903.574602:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345484424235 time 345484518881 state 0x0 ref 1 00000100:00100000:0.0:1789762903.574610:0:8057:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 8: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff91d400 gen 0 rate 20 depth 3 token 0 deadline 345484424235 time 345484518881 state 0x0 ref 1 00000100:00100000:0.0:1789762903.574636:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 3 deadline 376921712583 time 376821712583 state 0x1 ref 1 00000100:00100000:0.0:1789762903.574659:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 376921712583 time 376821736695 state 0x0 ref 1 00000100:00100000:0.0:1789762903.574667:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067498368:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762903.574684:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176f1a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.574697:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.574700:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.574703:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176f1a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.574711:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176f1a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.577850:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176f1a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.577861:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176f1a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.577867:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176f1a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.577873:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762903.577876:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176f1a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.577882:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176f1a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.577893:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176f1a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.577902:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f176f1a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762903.577963:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067498368:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3294us (3481us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762903.577970:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 824 00000100:00100000:0.0:1789762903.577977:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 376921712583 time 376821736695 state 0x2 ref 0 00000100:00100000:2.0:1789762903.579735:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762903.579771:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067498496 00000100:00100000:0.0:1789762903.579777:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067498496 00000100:00100000:0.0:1789762903.579801:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 376921712583 time 376821736695 state 0x0 ref 1 00000100:00100000:0.0:1789762903.579810:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 376921736695 time 376821736695 state 0x1 ref 1 00000100:00100000:0.0:1789762903.579833:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 376921736695 time 376826911640 state 0x0 ref 1 00000100:00100000:0.0:1789762903.579841:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067498496:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762903.579856:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176f21 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.579865:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.579868:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.579871:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176f21 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.579883:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176f21 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.583086:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176f21 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.583104:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176f21 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.583112:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176f21 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.583121:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762903.583126:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176f21 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.583135:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176f21 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.583154:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176f21 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.583163:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f176f21 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762903.583230:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067498496:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3386us (3492us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762903.583238:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 825 00000100:00100000:0.0:1789762903.583251:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 376921736695 time 376826911640 state 0x2 ref 0 00000100:00100000:1.0:1789762903.585437:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762903.585471:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067498624 00000100:00100000:0.0:1789762903.585478:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067498624 00000100:00100000:0.0:1789762903.585522:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 376921736695 time 376826911640 state 0x0 ref 1 00000100:00100000:0.0:1789762903.585532:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 376926911640 time 376826911640 state 0x1 ref 1 00000100:00100000:0.0:1789762903.585550:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376926911640 time 376832628411 state 0x0 ref 1 00000100:00100000:0.0:1789762903.585558:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067498624:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762903.585578:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f28 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.585588:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.585591:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.585596:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f28 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.585605:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f28 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.588262:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f28 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.588283:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f28 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.588293:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f28 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.588300:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762903.588303:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f28 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.588309:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f28 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.588322:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f28 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.588327:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f28 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762903.588370:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067498624:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2811us (2934us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762903.588377:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 826 00000100:00100000:0.0:1789762903.588383:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376926911640 time 376832628411 state 0x2 ref 0 00000100:00100000:2.0:1789762903.589986:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762903.590027:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067498752 00000100:00100000:3.0:1789762903.590032:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067498752 00000100:00100000:3.0:1789762903.590062:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376926911640 time 376832628411 state 0x0 ref 1 00000100:00100000:3.0:1789762903.590086:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376932628411 time 376832628411 state 0x1 ref 1 00010000:00010000:2.0:1789762903.680004:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762903.685639:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376932628411 time 376932707774 state 0x0 ref 1 00000100:00100000:0.0:1789762903.685664:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067498752:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762903.685698:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f2f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.685719:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.685722:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.685727:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f2f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.685737:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f2f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.688516:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f2f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.688530:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f2f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.688540:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f2f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.688547:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762903.688551:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f2f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.688564:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f2f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.688579:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f2f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.688585:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f2f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762903.688631:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067498752:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2976us (98644us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762903.688648:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 827 00000100:00100000:0.0:1789762903.688659:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376932628411 time 376932707774 state 0x2 ref 0 00000100:00100000:0.0:1789762903.690525:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762903.690570:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067498880 00000100:00100000:1.0:1789762903.690578:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067498880 00000100:00100000:1.0:1789762903.690614:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376932628411 time 376932707774 state 0x0 ref 1 00000100:00100000:1.0:1789762903.690625:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377032707774 time 376932707774 state 0x1 ref 1 00000100:00100000:3.0:1789762903.785737:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377032707774 time 377032799603 state 0x0 ref 1 00000100:00100000:3.0:1789762903.785763:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067498880:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762903.785799:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f36 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.785813:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762903.785817:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762903.785821:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f36 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.785833:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f36 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.788436:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f36 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.788445:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f36 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.788451:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f36 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.788457:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762903.788460:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f36 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.788466:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f36 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.788479:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f36 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762903.788484:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f36 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762903.788527:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067498880:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2773us (98003us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762903.788534:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 828 00000100:00100000:3.0:1789762903.788541:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377032707774 time 377032799603 state 0x2 ref 0 00000100:00100000:0.0:1789762903.790514:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762903.790590:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067499008 00000100:00100000:2.0:1789762903.790598:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067499008 00000100:00100000:2.0:1789762903.790635:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377032707774 time 377032799603 state 0x0 ref 1 00000100:00100000:2.0:1789762903.790646:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377132799603 time 377032799603 state 0x1 ref 1 00000100:00100000:0.0:1789762903.885770:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377132799603 time 377132839789 state 0x0 ref 1 00000100:00100000:0.0:1789762903.885792:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067499008:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762903.885828:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f3d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.885842:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.885845:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.885850:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f3d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.885866:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f3d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.888704:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f3d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.888723:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f3d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.888732:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f3d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.888740:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762903.888744:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f3d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.888754:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f3d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.888773:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f3d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762903.888782:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f176f3d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762903.888851:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067499008:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3064us (98337us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762903.888862:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 829 00000100:00100000:1.0:1789762903.888873:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377132799603 time 377132839789 state 0x2 ref 0 00000100:00100000:3.0:1789762903.890628:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762903.890671:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067499136 00000100:00100000:1.0:1789762903.890681:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067499136 00000100:00100000:1.0:1789762903.890723:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377132799603 time 377132839789 state 0x0 ref 1 00000100:00100000:1.0:1789762903.890737:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377232839789 time 377132839789 state 0x1 ref 1 00000100:00100000:0.0:1789762903.985845:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377232839789 time 377232911559 state 0x0 ref 1 00000100:00100000:0.0:1789762903.985868:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067499136:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762903.985904:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000601682fc/0xdf5a7ac59f176f44 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.985937:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.985942:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762903.985947:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000601682fc/0xdf5a7ac59f176f44 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.985960:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000601682fc/0xdf5a7ac59f176f44 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.988676:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000601682fc/0xdf5a7ac59f176f44 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.988689:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000601682fc/0xdf5a7ac59f176f44 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.988697:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000601682fc/0xdf5a7ac59f176f44 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.988707:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e0eaca00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762903.988711:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000601682fc/0xdf5a7ac59f176f44 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.988717:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000601682fc/0xdf5a7ac59f176f44 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.988729:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000601682fc/0xdf5a7ac59f176f44 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762903.988734:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000601682fc/0xdf5a7ac59f176f44 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762903.988779:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067499136:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2918us (98151us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762903.988789:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 830 00000100:00100000:0.0:1789762903.988807:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377232839789 time 377232911559 state 0x2 ref 0 00000100:00100000:2.0:1789762903.990585:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762903.990622:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067499264 00000100:00100000:3.0:1789762903.990631:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067499264 00000100:00100000:3.0:1789762903.990673:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377232839789 time 377232911559 state 0x0 ref 1 00000100:00100000:3.0:1789762903.990687:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377332911559 time 377232911559 state 0x1 ref 1 00000001:00080000:3.0:1789762904.073822:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967434 is committed 00000001:00080000:3.0:1789762904.073836:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967437 is committed 00000100:00100000:3.0:1789762904.085935:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377332911559 time 377332984288 state 0x0 ref 1 00000100:00100000:3.0:1789762904.085957:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067499264:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762904.085995:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f4b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.086011:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762904.086015:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762904.086022:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f4b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.086035:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f4b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.089335:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f4b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.089350:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f4b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.089360:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f4b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.089369:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762904.089375:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f4b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.089384:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f4b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.089403:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f4b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.089411:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f4b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762904.089469:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067499264:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3517us (98882us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762904.089478:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 831 00000100:00100000:3.0:1789762904.089489:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377332911559 time 377332984288 state 0x2 ref 0 00000100:00100000:2.0:1789762904.091410:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762904.091452:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067499392 00000100:00100000:1.0:1789762904.091461:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067499392 00000100:00100000:1.0:1789762904.091503:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377332911559 time 377332984288 state 0x0 ref 1 00000100:00100000:1.0:1789762904.091519:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377432984288 time 377332984288 state 0x1 ref 1 00000100:00100000:2.0:1789762904.186026:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377432984288 time 377433091861 state 0x0 ref 1 00000100:00100000:2.0:1789762904.186047:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067499392:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762904.186093:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f52 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.186107:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762904.186110:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762904.186114:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f52 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.186125:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f52 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.188885:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f52 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.188895:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f52 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.188901:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f52 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.188906:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762904.188909:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f52 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.188915:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f52 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.188945:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f52 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.188950:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f52 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762904.188995:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067499392:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2955us (97585us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762904.189003:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 832 00000100:00100000:2.0:1789762904.189016:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377432984288 time 377433091861 state 0x2 ref 0 00000100:00100000:2.0:1789762904.190777:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762904.190819:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067499520 00000100:00100000:0.0:1789762904.190828:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067499520 00000100:00100000:0.0:1789762904.190863:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377432984288 time 377433091861 state 0x0 ref 1 00000100:00100000:0.0:1789762904.190873:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377533091861 time 377433091861 state 0x1 ref 1 00000100:00100000:2.0:1789762904.286141:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377533091861 time 377533206533 state 0x0 ref 1 00000100:00100000:2.0:1789762904.286171:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067499520:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762904.286206:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f59 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.286220:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762904.286223:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762904.286228:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f59 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.286238:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f59 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.288903:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f59 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.288935:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f59 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.288943:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f59 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.288949:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762904.288953:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f59 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.288960:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f59 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.288975:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f59 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.288981:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f176f59 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762904.289029:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067499520:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2873us (98252us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762904.289037:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 833 00000100:00100000:0.0:1789762904.289045:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377533091861 time 377533206533 state 0x2 ref 0 00000100:00100000:0.0:1789762904.290899:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762904.290952:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067499648 00000100:00100000:3.0:1789762904.290961:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067499648 00000100:00100000:3.0:1789762904.291001:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377533091861 time 377533206533 state 0x0 ref 1 00000100:00100000:3.0:1789762904.291014:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377633206533 time 377533206533 state 0x1 ref 1 00000100:00100000:0.0:1789762904.386213:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377633206533 time 377633278348 state 0x0 ref 1 00000100:00100000:0.0:1789762904.386234:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067499648:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762904.386268:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f176f60 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.386283:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762904.386286:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762904.386290:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f176f60 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.386302:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f176f60 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.388987:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f176f60 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.388998:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f176f60 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.389006:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f176f60 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.389012:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ec00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762904.389015:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f176f60 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.389028:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f176f60 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.389049:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f176f60 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.389055:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f176f60 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762904.389116:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067499648:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2889us (98218us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762904.389123:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 834 00000100:00100000:1.0:1789762904.389131:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377633206533 time 377633278348 state 0x2 ref 0 00000100:00100000:2.0:1789762904.391132:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762904.391166:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067499776 00000100:00100000:1.0:1789762904.391175:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067499776 00000100:00100000:1.0:1789762904.391210:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377633206533 time 377633278348 state 0x0 ref 1 00000100:00100000:1.0:1789762904.391220:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377733278348 time 377633278348 state 0x1 ref 1 00000100:00100000:3.0:1789762904.486298:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377733278348 time 377733364738 state 0x0 ref 1 00000100:00100000:3.0:1789762904.486321:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067499776:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762904.486356:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f67 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.486369:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762904.486373:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762904.486377:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f67 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.486388:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f67 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.489520:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f67 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.489531:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f67 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.489538:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f67 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.489544:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762904.489548:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f67 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.489554:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f67 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.489572:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f67 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.489578:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f67 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762904.489636:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067499776:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3321us (98504us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762904.489644:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 835 00000100:00100000:3.0:1789762904.489652:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377733278348 time 377733364738 state 0x2 ref 0 00000100:00100000:2.0:1789762904.492057:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762904.492112:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067499904 00000100:00100000:0.0:1789762904.492120:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067499904 00000100:00100000:0.0:1789762904.492164:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377733278348 time 377733364738 state 0x0 ref 1 00000100:00100000:0.0:1789762904.492176:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377833364738 time 377733364738 state 0x1 ref 1 00000100:00100000:0.0:1789762904.519798:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762904.519830:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762904.519851:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762904.519853:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067500032 00000100:00100000:1.0:1789762904.519862:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067500032 00000100:00100000:3.0:1789762904.519871:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067500160 00000100:00100000:3.0:1789762904.519878:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067500160 00000100:00100000:0.0:1789762904.519892:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762904.519914:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 373 00000100:00100000:2.0:1789762904.519914:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067500288 00000100:00100000:3.0:1789762904.519946:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 558 00000100:00100000:3.0:1789762904.519957:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2397:x1876702067500160:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762904.519968:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067500288 00000100:00100000:1.0:1789762904.519974:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d1552450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067500032:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762904.519984:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762904.520010:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067500544 00000100:00100000:2.0:1789762904.520013:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067500544 00000100:00100000:3.0:1789762904.520033:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2397:x1876702067500160:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 78us (200us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762904.520044:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 558 00000100:00100000:1.0:1789762904.520045:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d1552450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067500032:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 71us (247us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762904.520051:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762904.520058:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 559 00000100:00100000:3.0:1789762904.520075:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067500672 00000100:00100000:1.0:1789762904.520078:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 373 00000100:00100000:3.0:1789762904.520079:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067500672 00000100:00100000:2.0:1789762904.520088:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c96a0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067500288:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762904.520111:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 560 00000100:00100000:3.0:1789762904.520121:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067500544:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762904.520123:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067500416 00000100:00100000:1.0:1789762904.520129:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067500416 00000100:00100000:2.0:1789762904.520131:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c96a0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067500288:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 48us (276us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762904.520151:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 559 00000100:00100000:3.0:1789762904.520156:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067500544:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 35us (170us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762904.520167:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 560 00000100:00100000:2.0:1789762904.520182:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 561 00000100:00100000:1.0:1789762904.520191:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 229 00000100:00100000:2.0:1789762904.520196:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067500672:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762904.520205:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067500416:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:0.0:1789762904.520218:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762904.520224:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762904.520227:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762904.520237:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067500672:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 46us (189us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762904.520243:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067500416:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 39us (351us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762904.520249:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 561 00000100:00100000:1.0:1789762904.520255:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 229 00000100:00100000:1.0:1789762904.586403:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377833364738 time 377833471188 state 0x0 ref 1 00000100:00100000:1.0:1789762904.586423:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067499904:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762904.586455:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176f6e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.586470:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762904.586473:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762904.586478:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176f6e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.586490:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176f6e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.589435:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176f6e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.589445:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176f6e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.589457:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176f6e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.589463:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762904.589467:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176f6e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.589473:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176f6e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.589485:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176f6e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.589490:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176f6e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762904.589534:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067499904:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3118us (97478us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762904.589542:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 836 00000100:00100000:1.0:1789762904.589549:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377833364738 time 377833471188 state 0x2 ref 0 00000100:00100000:3.0:1789762904.591760:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762904.591827:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067500800 00000100:00100000:1.0:1789762904.591835:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067500800 00000100:00100000:1.0:1789762904.591868:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377833364738 time 377833471188 state 0x0 ref 1 00000100:00100000:1.0:1789762904.591881:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377933471188 time 377833471188 state 0x1 ref 1 00000100:00100000:0.0:1789762904.686509:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377933471188 time 377933571714 state 0x0 ref 1 00000100:00100000:0.0:1789762904.686538:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067500800:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762904.686580:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176f75 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.686596:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762904.686600:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762904.686605:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176f75 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.686616:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176f75 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.689771:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176f75 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.689781:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176f75 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.689788:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176f75 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.689793:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762904.689798:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176f75 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.689805:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176f75 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.689819:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176f75 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.689824:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d22a0933/0xdf5a7ac59f176f75 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762904.689868:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067500800:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3342us (98109us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762904.689875:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 837 00000100:00100000:0.0:1789762904.689882:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377933471188 time 377933571714 state 0x2 ref 0 00000100:00100000:0.0:1789762904.692243:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762904.692283:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067500928 00000100:00100000:3.0:1789762904.692291:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067500928 00000100:00100000:3.0:1789762904.692327:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 377933471188 time 377933571714 state 0x0 ref 1 00000100:00100000:3.0:1789762904.692338:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378033571714 time 377933571714 state 0x1 ref 1 00010000:00010000:3.0:1789762904.704047:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762904.786560:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378033571714 time 378033625256 state 0x0 ref 1 00000100:00100000:3.0:1789762904.786582:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067500928:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762904.786616:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f7c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.786630:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762904.786634:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762904.786639:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f7c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.786651:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f7c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.789828:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f7c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.789841:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f7c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.789848:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f7c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.789853:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762904.789857:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f7c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.789864:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f7c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.789878:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f7c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762904.789883:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f7c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762904.789954:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067500928:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3380us (97710us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762904.789962:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 838 00000100:00100000:3.0:1789762904.789970:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378033571714 time 378033625256 state 0x2 ref 0 00000100:00100000:0.0:1789762904.792384:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762904.792427:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067501056 00000100:00100000:1.0:1789762904.792435:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067501056 00000100:00100000:1.0:1789762904.792472:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378033571714 time 378033625256 state 0x0 ref 1 00000100:00100000:1.0:1789762904.792482:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378133625256 time 378033625256 state 0x1 ref 1 00000100:00100000:2.0:1789762904.886629:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378133625256 time 378133695953 state 0x0 ref 1 00000100:00100000:2.0:1789762904.886648:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067501056:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762904.886679:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176f83 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.886692:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762904.886695:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762904.886699:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176f83 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762904.886710:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176f83 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.890151:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176f83 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.890163:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176f83 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.890171:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176f83 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.890177:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c762d000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762904.890180:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176f83 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.890195:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176f83 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.890210:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176f83 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.890217:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd89f469/0xdf5a7ac59f176f83 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762904.890264:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067501056:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3622us (97881us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762904.890270:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 839 00000100:00100000:1.0:1789762904.890279:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378133625256 time 378133695953 state 0x2 ref 0 00000100:00100000:0.0:1789762904.892608:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762904.892651:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067501184 00000100:00100000:2.0:1789762904.892659:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067501184 00000100:00100000:2.0:1789762904.892694:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378133625256 time 378133695953 state 0x0 ref 1 00000100:00100000:2.0:1789762904.892712:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378233695953 time 378133695953 state 0x1 ref 1 00000100:00100000:0.0:1789762904.986710:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378233695953 time 378233775468 state 0x0 ref 1 00000100:00100000:0.0:1789762904.986733:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d204e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067501184:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762904.986767:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176f8a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.986785:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762904.986789:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762904.986793:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176f8a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762904.986804:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176f8a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.990108:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176f8a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.990119:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176f8a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.990127:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176f8a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.990133:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762904.990137:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176f8a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.990143:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176f8a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.990157:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176f8a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762904.990164:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f176f8a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762904.990219:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d204e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067501184:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3493us (97612us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762904.990227:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 840 00000100:00100000:1.0:1789762904.990235:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378233695953 time 378233775468 state 0x2 ref 0 00000100:00100000:2.0:1789762904.992826:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762904.992861:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067501312 00000100:00100000:3.0:1789762904.992869:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067501312 00000100:00100000:3.0:1789762904.992905:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378233695953 time 378233775468 state 0x0 ref 1 00000100:00100000:3.0:1789762904.992933:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378333775468 time 378233775468 state 0x1 ref 1 00000100:00100000:0.0:1789762905.086804:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378333775468 time 378333867884 state 0x0 ref 1 00000100:00100000:0.0:1789762905.086854:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067501312:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:2.0:1789762905.086884:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762905.086899:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176f91 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.086943:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762905.086949:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762905.086956:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176f91 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.086976:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176f91 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.091769:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176f91 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.091781:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176f91 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.091788:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176f91 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.091794:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762905.091798:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176f91 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.091805:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176f91 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.091818:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176f91 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.091824:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020398f3e/0xdf5a7ac59f176f91 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762905.091874:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067501312:12345-192.168.203.37@tcp:3:dd.500 Request processed in 5053us (99048us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762905.091885:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 841 00000100:00100000:1.0:1789762905.091896:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378333775468 time 378333867884 state 0x2 ref 0 00000100:00100000:2.0:1789762905.093894:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762905.093953:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067501440 00000100:00100000:1.0:1789762905.093961:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067501440 00000100:00100000:1.0:1789762905.093997:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378333775468 time 378333867884 state 0x0 ref 1 00000100:00100000:1.0:1789762905.094008:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378433867884 time 378333867884 state 0x1 ref 1 00000100:00100000:3.0:1789762905.186894:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378433867884 time 378433962000 state 0x0 ref 1 00000100:00100000:3.0:1789762905.186913:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067501440:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762905.186964:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f98 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.186977:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762905.186981:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762905.186986:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f98 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.186996:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f98 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.189519:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f98 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.189535:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f98 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.189546:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f98 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.189555:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762905.189569:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f98 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.189580:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f98 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.189594:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f98 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.189600:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000054047fbc/0xdf5a7ac59f176f98 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762905.189654:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067501440:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2746us (95759us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762905.189665:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 842 00000100:00100000:1.0:1789762905.189673:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378433867884 time 378433962000 state 0x2 ref 0 00000100:00100000:2.0:1789762905.191485:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762905.191524:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067501568 00000100:00100000:1.0:1789762905.191532:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067501568 00000100:00100000:1.0:1789762905.191566:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378433867884 time 378433962000 state 0x0 ref 1 00000100:00100000:1.0:1789762905.191575:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378533962000 time 378433962000 state 0x1 ref 1 00000100:00100000:3.0:1789762905.286998:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378533962000 time 378534067371 state 0x0 ref 1 00000100:00100000:3.0:1789762905.287018:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067501568:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762905.287046:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176f9f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.287079:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762905.287082:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762905.287088:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176f9f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.287098:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176f9f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.289708:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176f9f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.289724:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176f9f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.289736:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176f9f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.289745:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762905.289750:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176f9f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.289761:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176f9f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.289779:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176f9f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.289791:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176f9f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762905.289836:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067501568:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2824us (98351us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762905.289843:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 843 00000100:00100000:3.0:1789762905.289851:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378533962000 time 378534067371 state 0x2 ref 0 00000100:00100000:0.0:1789762905.291599:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762905.291641:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067501696 00000100:00100000:1.0:1789762905.291650:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067501696 00000100:00100000:1.0:1789762905.291684:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378533962000 time 378534067371 state 0x0 ref 1 00000100:00100000:1.0:1789762905.291693:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378634067371 time 378534067371 state 0x1 ref 1 00000100:00100000:2.0:1789762905.387101:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378634067371 time 378634164233 state 0x0 ref 1 00000100:00100000:2.0:1789762905.387130:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a11180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067501696:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762905.387171:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f176fa6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762905.387186:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762905.387191:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762905.387197:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f176fa6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762905.387212:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f176fa6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.390148:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f176fa6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.390167:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f176fa6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.390173:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f176fa6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.390180:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f0400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762905.390183:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f176fa6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.390190:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f176fa6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.390205:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f176fa6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.390211:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c5e4ee2/0xdf5a7ac59f176fa6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762905.390259:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a11180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067501696:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3141us (98661us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762905.390267:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 844 00000100:00100000:0.0:1789762905.390275:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378634067371 time 378634164233 state 0x2 ref 0 00000100:00100000:0.0:1789762905.392190:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762905.392234:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067501824 00000100:00100000:1.0:1789762905.392243:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067501824 00000100:00100000:1.0:1789762905.392281:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378634067371 time 378634164233 state 0x0 ref 1 00000100:00100000:1.0:1789762905.392292:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378734164233 time 378634164233 state 0x1 ref 1 00000100:00100000:0.0:1789762905.487199:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378734164233 time 378734266263 state 0x0 ref 1 00000100:00100000:0.0:1789762905.487222:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067501824:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762905.487256:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176fad lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.487270:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762905.487274:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762905.487278:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176fad lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.487289:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176fad lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.490134:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176fad lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.490146:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176fad lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.490152:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176fad lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.490158:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762905.490162:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176fad lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.490173:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176fad lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.490188:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176fad lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.490193:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f176fad lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762905.490239:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067501824:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3026us (98049us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762905.490246:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 845 00000100:00100000:0.0:1789762905.490254:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378734164233 time 378734266263 state 0x2 ref 0 00000100:00100000:0.0:1789762905.492127:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762905.492167:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067501952 00000100:00100000:3.0:1789762905.492176:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067501952 00000100:00100000:3.0:1789762905.492221:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378734164233 time 378734266263 state 0x0 ref 1 00000100:00100000:3.0:1789762905.492234:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378834266263 time 378734266263 state 0x1 ref 1 00000100:00100000:2.0:1789762905.587279:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378834266263 time 378834342950 state 0x0 ref 1 00000100:00100000:2.0:1789762905.587311:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d2a10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067501952:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762905.587354:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f176fb4 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762905.587373:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762905.587391:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762905.587396:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f176fb4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762905.587407:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f176fb4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.590093:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f176fb4 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.590105:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f176fb4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.590112:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f176fb4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.590118:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998fc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762905.590122:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f176fb4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.590129:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f176fb4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.590143:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f176fb4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.590149:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f176fb4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762905.590204:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d2a10000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067501952:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2906us (98075us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762905.590215:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 846 00000100:00100000:1.0:1789762905.590225:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378834266263 time 378834342950 state 0x2 ref 0 00000100:00100000:0.0:1789762905.592026:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762905.592109:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067502080 00000100:00100000:2.0:1789762905.592118:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067502080 00000100:00100000:2.0:1789762905.592161:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378834266263 time 378834342950 state 0x0 ref 1 00000100:00100000:2.0:1789762905.592177:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378934342950 time 378834342950 state 0x1 ref 1 00000100:00100000:0.0:1789762905.600074:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8a8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074725248:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762905.600079:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fd9f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074725376:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762905.600088:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6ba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074725504:0@lo:400:kworker.0 00000100:00100000:0.0:1789762905.600128:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762905.600145:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762905.600169:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762905.600177:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074725632:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762905.600190:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6ba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074725760:0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.600201:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6b9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074726016:0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.600207:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074725888:0@lo:400:kworker.0 00000100:00100000:3.0:1789762905.600218:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762905.600227:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762905.600251:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074726144:0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.600260:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762905.600261:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762905.600272:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff6b9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074726272:0@lo:400:kworker.0 00000100:00100000:3.0:1789762905.600291:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074725504 00000100:00100000:3.0:1789762905.600297:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074725504 00000100:00100000:1.0:1789762905.600304:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff685180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074726528:0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.600321:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff685500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074726400:0@lo:400:kworker.0 00000100:00100000:0.0:1789762905.600322:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762905.600339:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 374 00000100:00100000:3.0:1789762905.600350:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb939450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074725504:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.600368:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074725248 00000100:00100000:0.0:1789762905.600370:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff685880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074726784:0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.600374:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074725248 00000100:00100000:2.0:1789762905.600375:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762905.600393:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb939450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074725504:12345-0@lo:400:kworker.0 Request processed in 46us (249us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762905.600399:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 374 00000100:00100000:2.0:1789762905.600412:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff684380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074726912:0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.600418:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074725376 00000100:00100000:0.0:1789762905.600420:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762905.600422:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074725376 00000100:00100000:3.0:1789762905.600436:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074725632 00000100:00100000:3.0:1789762905.600441:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074725632 00000100:00100000:2.0:1789762905.600449:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762905.600456:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:1.0:1789762905.600467:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074725248:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762905.600473:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8a9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074727296:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762905.600473:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 231 00000100:00100000:3.0:1789762905.600486:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074725376:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789762905.600492:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762905.600499:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074726784 00000100:00100000:2.0:1789762905.600506:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074726784 00002000:00000024:3.0:1789762905.600506:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762905.600515:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762905.600544:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074725248:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (415us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762905.600548:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074726912 00000100:00100000:1.0:1789762905.600553:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:2.0:1789762905.600553:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074726912 00000100:00100000:3.0:1789762905.600572:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074725376:12345-0@lo:13:osp-pre-1-1.0 Request processed in 87us (405us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762905.600574:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8a8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074725248:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762905.600576:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074727296 00000100:00100000:1.0:1789762905.600581:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074727296 00000100:00100000:3.0:1789762905.600587:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 231 00000100:00100000:2.0:1789762905.600588:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:2.0:1789762905.600602:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074726784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762905.600614:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00100000:1.0:1789762905.600621:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 233 00000100:00100000:3.0:1789762905.600625:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074725632:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762905.600631:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 231 00000100:00100000:1.0:1789762905.600634:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074727296:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789762905.600640:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762905.600645:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074726912:12345-0@lo:400:kworker.0 00002000:00000024:1.0:1789762905.600646:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762905.600658:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074726784:12345-0@lo:400:kworker.0 Request processed in 60us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762905.600666:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:3.0:1789762905.600684:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff686d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074725632:12345-0@lo:13:osp-pre-0-1.0 Request processed in 61us (464us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762905.600692:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074727296:12345-0@lo:13:osp-pre-1-0.0 Request processed in 61us (177us total) trans 0 rc 0/0 00000100:00080000:2.0:1789762905.600693:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762905.600693:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00080000:2.0:1789762905.600701:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762905.600705:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 233 00000100:00100000:0.0:1789762905.600712:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074726912:12345-0@lo:400:kworker.0 Request processed in 71us (262us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762905.600719:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 231 00000100:00100000:3.0:1789762905.600733:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074725760 00000100:00100000:2.0:1789762905.600735:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074726016 00000100:00100000:3.0:1789762905.600738:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074725760 00000100:00100000:2.0:1789762905.600749:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074726016 00000100:00100000:1.0:1789762905.600769:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074726272 00000100:00100000:3.0:1789762905.600769:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074726400 00000100:00100000:0.0:1789762905.600774:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8a9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074727296:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762905.600775:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074726400 00000100:00100000:1.0:1789762905.600777:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074726272 00000100:00100000:2.0:1789762905.600784:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 562 00000100:00100000:0.0:1789762905.600796:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff685880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074726784:0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.600797:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074725760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762905.600820:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 563 00000100:00100000:1.0:1789762905.600828:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 564 00000100:00100000:3.0:1789762905.600832:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074726016:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.600845:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074726400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.600851:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074725760:12345-0@lo:400:kworker.0 Request processed in 58us (623us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762905.600852:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762905.600864:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 562 00000100:00100000:2.0:1789762905.600881:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074726528 00000100:00100000:0.0:1789762905.600882:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff687800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074727040:0@lo:400:kworker.0 00000100:00100000:3.0:1789762905.600882:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074726016:12345-0@lo:400:kworker.0 Request processed in 51us (623us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762905.600887:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074726528 00000100:00100000:3.0:1789762905.600896:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 563 00000100:00100000:1.0:1789762905.600901:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074726400:12345-0@lo:400:kworker.0 Request processed in 60us (529us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762905.600908:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 564 00000100:00100000:3.0:1789762905.600943:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 565 00000100:00100000:0.0:1789762905.600947:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762905.600952:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 566 00000100:00100000:2.0:1789762905.600965:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074726528:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762905.600970:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074726272:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762905.600976:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6ba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074725504:0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.600996:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6ba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074725760:0@lo:400:kworker.0 00000100:00100000:0.0:1789762905.600999:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6b9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074726016:0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.601024:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3427:x1876702074726528:12345-0@lo:400:kworker.0 Request processed in 62us (173us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762905.601028:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074726272:12345-0@lo:400:kworker.0 Request processed in 73us (707us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762905.601033:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6b9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074726272:0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.601037:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 566 00000100:00100000:0.0:1789762905.601038:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff685180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3427:1876702074726528:0@lo:400:kworker.0 00000100:00100000:3.0:1789762905.601052:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 565 00000100:00100000:1.0:1789762905.601086:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074727040 00000100:00100000:1.0:1789762905.601090:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074727040 00000100:00100000:2.0:1789762905.601098:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074725888 00000100:00100000:2.0:1789762905.601103:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074725888 00000100:00100000:3.0:1789762905.601123:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762905.601127:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00100000:0.0:1789762905.601138:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074727040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.601144:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074726144 00000100:00100000:2.0:1789762905.601150:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074726144 00000100:00100000:3.0:1789762905.601164:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff686a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074726656:0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.601178:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff685500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074726400:0@lo:400:kworker.0 00000100:00100000:0.0:1789762905.601187:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074727040:12345-0@lo:400:kworker.0 Request processed in 51us (267us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762905.601190:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:1.0:1789762905.601198:0:3426:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294967417 gen 2 00000100:00100000:0.0:1789762905.601202:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00100000:3.0:1789762905.601206:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:1.0:1789762905.601210:0:3426:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37e3bdc000, transno 4294967417 00000100:00100000:2.0:1789762905.601210:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074725888:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762905.601231:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff686680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074727168:0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.601232:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff684380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074726912:0@lo:400:kworker.0 00000100:00100000:0.0:1789762905.601242:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff687800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074727040:0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.601261:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fd9f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074725376:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762905.601271:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762905.601277:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074725888:12345-0@lo:400:kworker.0 Request processed in 72us (1016us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762905.601281:0:3426:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967434 gen 2 00000100:00100000:0.0:1789762905.601285:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000004:00080000:1.0:1789762905.601288:0:3426:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967433 committed transno 4294967434 00000100:00100000:0.0:1789762905.601297:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074726144:12345-0@lo:400:kworker.0 00000004:00080000:1.0:1789762905.601302:0:3426:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967434 committed transno 4294967434 00000100:00100000:2.0:1789762905.601313:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:1.0:1789762905.601318:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074725888:0@lo:400:kworker.0 00000100:00100000:3.0:1789762905.601330:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074725632:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762905.601345:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff685180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074726144:12345-0@lo:400:kworker.0 Request processed in 50us (224us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762905.601351:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff6bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074726144:0@lo:400:kworker.0 00000100:00100000:0.0:1789762905.601355:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00000100:00100000:2.0:1789762905.601376:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074726656 00000100:00080000:0.0:1789762905.601383:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762905.601384:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074727168 00000100:00100000:2.0:1789762905.601384:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074726656 00000100:00080000:0.0:1789762905.601390:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762905.601391:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074727168 00000100:00100000:0.0:1789762905.601426:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 233 00000100:00100000:0.0:1789762905.601437:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff687800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074727168:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.601446:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 567 00000100:00080000:1.0:1789762905.601447:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762905.601459:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff685880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074726656:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762905.601486:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff687800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074727168:12345-0@lo:400:kworker.0 Request processed in 51us (218us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762905.601495:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 233 00000100:00100000:2.0:1789762905.601508:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff685880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3424:x1876702074726656:12345-0@lo:400:kworker.0 Request processed in 50us (307us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762905.601519:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 567 00000100:00100000:0.0:1789762905.601539:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff686a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3424:1876702074726656:0@lo:400:kworker.0 00000100:00100000:0.0:1789762905.601557:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff686680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074727168:0@lo:400:kworker.0 00000100:00100000:0.0:1789762905.687417:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378934342950 time 378934482700 state 0x0 ref 1 00000100:00100000:0.0:1789762905.687444:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067502080:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762905.687481:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176fbb lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.687495:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762905.687499:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762905.687504:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176fbb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.687515:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176fbb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.690114:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176fbb lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.690128:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176fbb lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.690135:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176fbb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.690141:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762905.690145:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176fbb lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.690151:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176fbb lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.690165:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176fbb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762905.690170:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f176fbb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762905.690221:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067502080:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2788us (98196us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762905.690228:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 847 00000100:00100000:0.0:1789762905.690236:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378934342950 time 378934482700 state 0x2 ref 0 00000100:00100000:0.0:1789762905.692116:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762905.692170:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067502208 00000100:00100000:1.0:1789762905.692180:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067502208 00000100:00100000:1.0:1789762905.692217:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 378934342950 time 378934482700 state 0x0 ref 1 00000100:00100000:1.0:1789762905.692228:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379034482700 time 378934482700 state 0x1 ref 1 00010000:00010000:0.0:1789762905.728019:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762905.787487:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379034482700 time 379034553535 state 0x0 ref 1 00000100:00100000:1.0:1789762905.787514:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067502208:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762905.787552:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.787565:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762905.787577:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762905.787582:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.787594:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.790316:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.790327:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.790335:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.790341:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762905.790345:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.790351:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.790366:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.790380:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762905.790430:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067502208:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2928us (98315us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762905.790438:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 848 00000100:00100000:1.0:1789762905.790452:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379034482700 time 379034553535 state 0x2 ref 0 00000100:00100000:0.0:1789762905.792316:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762905.792356:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067502336 00000100:00100000:1.0:1789762905.792364:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067502336 00000100:00100000:1.0:1789762905.792398:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379034482700 time 379034553535 state 0x0 ref 1 00000100:00100000:1.0:1789762905.792412:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379134553535 time 379034553535 state 0x1 ref 1 00000100:00100000:1.0:1789762905.887577:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379134553535 time 379134635286 state 0x0 ref 1 00000100:00100000:1.0:1789762905.887604:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067502336:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762905.887638:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc9 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.887652:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762905.887655:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762905.887660:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.887672:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.890466:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.890478:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.890486:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.890500:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762905.890505:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.890515:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.890531:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762905.890538:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fc9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762905.890604:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067502336:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3010us (98286us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762905.890614:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 849 00000100:00100000:1.0:1789762905.890625:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379134553535 time 379134635286 state 0x2 ref 0 00000100:00100000:2.0:1789762905.892504:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762905.892562:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067502464 00000100:00100000:0.0:1789762905.892571:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067502464 00000100:00100000:0.0:1789762905.892618:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379134553535 time 379134635286 state 0x0 ref 1 00000100:00100000:0.0:1789762905.892635:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379234635286 time 379134635286 state 0x1 ref 1 00000100:00100000:3.0:1789762905.987684:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379234635286 time 379234744510 state 0x0 ref 1 00000100:00100000:3.0:1789762905.987711:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9247800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067502464:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762905.987753:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176fd0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.987770:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762905.987774:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762905.987779:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176fd0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.987793:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176fd0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.990731:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176fd0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.990748:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176fd0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.990755:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176fd0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.990761:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762905.990765:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176fd0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.990771:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176fd0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.990785:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176fd0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762905.990790:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f176fd0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762905.990837:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9247800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067502464:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3136us (98333us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762905.990844:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 850 00000100:00100000:3.0:1789762905.990852:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379234635286 time 379234744510 state 0x2 ref 0 00000100:00100000:2.0:1789762905.992698:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762905.992740:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067502592 00000100:00100000:1.0:1789762905.992747:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067502592 00000100:00100000:1.0:1789762905.992790:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379234635286 time 379234744510 state 0x0 ref 1 00000100:00100000:1.0:1789762905.992806:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379334744510 time 379234744510 state 0x1 ref 1 00000100:00100000:2.0:1789762906.087790:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379334744510 time 379334854580 state 0x0 ref 1 00000100:00100000:2.0:1789762906.087815:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9247480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067502592:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762906.087852:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fd7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.087870:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762906.087875:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762906.087882:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fd7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.087897:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fd7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.090661:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fd7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.090685:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fd7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.090695:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fd7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.090703:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a6400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762906.090708:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fd7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.090719:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fd7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.090738:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fd7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.090747:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fd7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762906.090802:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9247480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067502592:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2996us (98103us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762906.090811:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 851 00000100:00100000:2.0:1789762906.090823:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379334744510 time 379334854580 state 0x2 ref 0 00000100:00100000:2.0:1789762906.092849:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762906.092950:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067502720 00000100:00100000:1.0:1789762906.092959:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067502720 00000100:00100000:1.0:1789762906.093002:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379334744510 time 379334854580 state 0x0 ref 1 00000100:00100000:1.0:1789762906.093017:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379434854580 time 379334854580 state 0x1 ref 1 00000100:00100000:1.0:1789762906.187879:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379434854580 time 379434941145 state 0x0 ref 1 00000100:00100000:1.0:1789762906.187950:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067502720:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762906.187996:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fde lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.188010:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762906.188014:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762906.188019:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fde lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.188030:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fde lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.191202:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fde lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.191218:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fde lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.191229:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fde lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.191238:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762906.191243:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fde lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.191255:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fde lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.191273:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fde lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.191281:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176fde lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762906.191334:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067502720:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3431us (98484us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762906.191343:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 852 00000100:00100000:1.0:1789762906.191352:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379434854580 time 379434941145 state 0x2 ref 0 00000100:00100000:2.0:1789762906.193301:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762906.193338:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067502848 00000100:00100000:3.0:1789762906.193346:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067502848 00000100:00100000:3.0:1789762906.193381:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379434854580 time 379434941145 state 0x0 ref 1 00000100:00100000:3.0:1789762906.193391:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379534941145 time 379434941145 state 0x1 ref 1 00000100:00100000:2.0:1789762906.287995:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379534941145 time 379535052995 state 0x0 ref 1 00000100:00100000:2.0:1789762906.288021:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067502848:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762906.288071:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fe5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.288085:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762906.288089:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762906.288094:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fe5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.288105:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fe5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.291089:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fe5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.291101:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fe5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.291108:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fe5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.291114:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a6400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762906.291118:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fe5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.291136:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fe5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.291150:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fe5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.291154:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f176fe5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762906.291200:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067502848:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3189us (97899us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762906.291207:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 853 00000100:00100000:2.0:1789762906.291216:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379534941145 time 379535052995 state 0x2 ref 0 00000100:00100000:2.0:1789762906.293041:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762906.293103:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067502976 00000100:00100000:0.0:1789762906.293112:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067502976 00000100:00100000:0.0:1789762906.293151:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379534941145 time 379535052995 state 0x0 ref 1 00000100:00100000:0.0:1789762906.293162:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379635052995 time 379535052995 state 0x1 ref 1 00000100:00100000:0.0:1789762906.388085:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379635052995 time 379635150528 state 0x0 ref 1 00000100:00100000:0.0:1789762906.388110:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067502976:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762906.388150:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f176fec lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.388164:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762906.388167:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762906.388172:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f176fec lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.388183:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f176fec lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.391017:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f176fec lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.391029:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f176fec lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.391036:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f176fec lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.391043:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762906.391047:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f176fec lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.391072:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f176fec lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.391092:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f176fec lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.391102:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f176fec lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762906.391161:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067502976:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3060us (98119us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762906.391173:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 854 00000100:00100000:1.0:1789762906.391184:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379635052995 time 379635150528 state 0x2 ref 0 00000100:00100000:0.0:1789762906.392984:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762906.393018:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067503104 00000100:00100000:1.0:1789762906.393026:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067503104 00000100:00100000:1.0:1789762906.393080:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379635052995 time 379635150528 state 0x0 ref 1 00000100:00100000:1.0:1789762906.393092:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379735150528 time 379635150528 state 0x1 ref 1 00000100:00100000:1.0:1789762906.488149:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379735150528 time 379735213501 state 0x0 ref 1 00000100:00100000:1.0:1789762906.488176:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067503104:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762906.488213:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ff3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.488227:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762906.488231:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762906.488236:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ff3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.488247:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ff3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.490952:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ff3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.490968:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ff3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.490977:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ff3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.490984:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762906.490989:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ff3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.490999:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ff3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.491018:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ff3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.491027:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ff3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762906.491103:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067503104:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2937us (98118us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762906.491113:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 855 00000100:00100000:1.0:1789762906.491122:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379735150528 time 379735213501 state 0x2 ref 0 00000100:00100000:2.0:1789762906.493141:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762906.493186:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067503232 00000100:00100000:1.0:1789762906.493196:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067503232 00000100:00100000:1.0:1789762906.493238:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379735150528 time 379735213501 state 0x0 ref 1 00000100:00100000:1.0:1789762906.493252:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379835213501 time 379735213501 state 0x1 ref 1 00000100:00100000:1.0:1789762906.588217:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379835213501 time 379835282261 state 0x0 ref 1 00000100:00100000:1.0:1789762906.588244:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067503232:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762906.588280:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ffa lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.588294:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762906.588298:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762906.588303:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ffa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.588314:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ffa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.591173:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ffa lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.591186:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ffa lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.591194:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ffa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.591200:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762906.591203:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ffa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.591210:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ffa lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.591223:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ffa lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.591229:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f176ffa lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762906.591284:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067503232:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3052us (98142us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762906.591292:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 856 00000100:00100000:1.0:1789762906.591300:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379835213501 time 379835282261 state 0x2 ref 0 00000100:00100000:3.0:1789762906.593164:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762906.593199:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067503360 00000100:00100000:1.0:1789762906.593207:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067503360 00000100:00100000:1.0:1789762906.593243:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379835213501 time 379835282261 state 0x0 ref 1 00000100:00100000:1.0:1789762906.593254:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379935282261 time 379835282261 state 0x1 ref 1 00000100:00100000:3.0:1789762906.688329:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379935282261 time 379935390535 state 0x0 ref 1 00000100:00100000:3.0:1789762906.688360:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff686680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067503360:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762906.688401:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f177001 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.688418:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762906.688422:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762906.688428:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f177001 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.688442:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f177001 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.691296:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f177001 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.691312:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f177001 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.691329:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f177001 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.691338:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762906.691343:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f177001 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.691352:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f177001 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.691369:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f177001 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.691378:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0be31cc/0xdf5a7ac59f177001 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762906.691433:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff686680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067503360:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3085us (98269us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762906.691445:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 857 00000100:00100000:3.0:1789762906.691456:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379935282261 time 379935390535 state 0x2 ref 0 00000100:00100000:3.0:1789762906.693406:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762906.693448:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067503488 00000100:00100000:1.0:1789762906.693455:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067503488 00000100:00100000:1.0:1789762906.693493:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 379935282261 time 379935390535 state 0x0 ref 1 00000100:00100000:1.0:1789762906.693504:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380035390535 time 379935390535 state 0x1 ref 1 00010000:00010000:1.0:1789762906.752071:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762906.788413:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380035390535 time 380035479450 state 0x0 ref 1 00000100:00100000:0.0:1789762906.788437:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067503488:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762906.788473:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006e38292d/0xdf5a7ac59f177008 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.788486:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762906.788490:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762906.788495:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e38292d/0xdf5a7ac59f177008 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.788506:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e38292d/0xdf5a7ac59f177008 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.791300:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006e38292d/0xdf5a7ac59f177008 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.791312:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e38292d/0xdf5a7ac59f177008 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.791319:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e38292d/0xdf5a7ac59f177008 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.791335:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e0ef4e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762906.791338:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e38292d/0xdf5a7ac59f177008 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.791345:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e38292d/0xdf5a7ac59f177008 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.791359:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e38292d/0xdf5a7ac59f177008 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.791364:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e38292d/0xdf5a7ac59f177008 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762906.791414:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff686a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067503488:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2986us (98009us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762906.791421:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 858 00000100:00100000:2.0:1789762906.791428:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380035390535 time 380035479450 state 0x2 ref 0 00000100:00100000:0.0:1789762906.793258:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762906.793307:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067503616 00000100:00100000:1.0:1789762906.793315:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067503616 00000100:00100000:1.0:1789762906.793352:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380035390535 time 380035479450 state 0x0 ref 1 00000100:00100000:1.0:1789762906.793363:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380135479450 time 380035479450 state 0x1 ref 1 00000100:00100000:0.0:1789762906.888524:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380135479450 time 380135582680 state 0x0 ref 1 00000100:00100000:0.0:1789762906.888549:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067503616:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762906.888583:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007d86533f/0xdf5a7ac59f17700f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.888597:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762906.888602:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762906.888608:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d86533f/0xdf5a7ac59f17700f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.888623:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d86533f/0xdf5a7ac59f17700f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.891559:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007d86533f/0xdf5a7ac59f17700f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.891570:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d86533f/0xdf5a7ac59f17700f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.891578:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d86533f/0xdf5a7ac59f17700f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.891583:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e0ef4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762906.891587:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d86533f/0xdf5a7ac59f17700f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.891595:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d86533f/0xdf5a7ac59f17700f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.891610:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d86533f/0xdf5a7ac59f17700f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.891617:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d86533f/0xdf5a7ac59f17700f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762906.891675:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067503616:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3135us (98417us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762906.891682:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 859 00000100:00100000:0.0:1789762906.891689:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380135479450 time 380135582680 state 0x2 ref 0 00000100:00100000:0.0:1789762906.893649:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762906.893688:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067503744 00000100:00100000:3.0:1789762906.893697:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067503744 00000100:00100000:3.0:1789762906.893742:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380135479450 time 380135582680 state 0x0 ref 1 00000100:00100000:3.0:1789762906.893758:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380235582680 time 380135582680 state 0x1 ref 1 00000100:00100000:2.0:1789762906.988599:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380235582680 time 380235664763 state 0x0 ref 1 00000100:00100000:2.0:1789762906.988623:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067503744:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762906.988660:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f177016 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.988688:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762906.988692:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762906.988697:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f177016 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.988708:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f177016 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.991704:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f177016 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.991720:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f177016 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.991731:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f177016 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.991740:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762906.991746:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f177016 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.991757:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f177016 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.991777:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f177016 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762906.991800:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f177016 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762906.991858:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067503744:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3242us (98208us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762906.991869:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 860 00000100:00100000:2.0:1789762906.991880:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380235582680 time 380235664763 state 0x2 ref 0 00000100:00100000:3.0:1789762906.993678:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762906.993726:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067503872 00000100:00100000:2.0:1789762906.993735:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067503872 00000100:00100000:2.0:1789762906.993778:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380235582680 time 380235664763 state 0x0 ref 1 00000100:00100000:2.0:1789762906.993795:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380335664763 time 380235664763 state 0x1 ref 1 00000100:00100000:0.0:1789762907.088720:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380335664763 time 380335785812 state 0x0 ref 1 00000100:00100000:0.0:1789762907.088758:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067503872:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:1.0:1789762907.088793:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762907.088800:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000045631fbc/0xdf5a7ac59f17701d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.088815:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762907.088818:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762907.088824:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045631fbc/0xdf5a7ac59f17701d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.088835:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045631fbc/0xdf5a7ac59f17701d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.091445:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000045631fbc/0xdf5a7ac59f17701d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.091455:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045631fbc/0xdf5a7ac59f17701d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.091462:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045631fbc/0xdf5a7ac59f17701d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.091467:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8c83a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762907.091470:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045631fbc/0xdf5a7ac59f17701d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.091477:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045631fbc/0xdf5a7ac59f17701d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.091490:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045631fbc/0xdf5a7ac59f17701d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.091495:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045631fbc/0xdf5a7ac59f17701d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.091544:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff684380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067503872:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2809us (97866us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762907.091552:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 861 00000100:00100000:1.0:1789762907.091561:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380335664763 time 380335785812 state 0x2 ref 0 00000100:00100000:3.0:1789762907.093650:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762907.093701:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067504000 00000100:00100000:0.0:1789762907.093712:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067504000 00000100:00100000:0.0:1789762907.093754:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380335664763 time 380335785812 state 0x0 ref 1 00000100:00100000:0.0:1789762907.093769:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380435785812 time 380335785812 state 0x1 ref 1 00000100:00100000:1.0:1789762907.188825:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380435785812 time 380435888207 state 0x0 ref 1 00000100:00100000:1.0:1789762907.188852:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff685500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067504000:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762907.188888:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177024 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.188902:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762907.188905:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762907.188909:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177024 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.188945:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177024 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.192063:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177024 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.192073:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177024 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.192079:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177024 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.192085:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762907.192090:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177024 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.192096:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177024 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.192110:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177024 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.192116:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177024 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.192163:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff685500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067504000:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3322us (98513us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762907.192169:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 862 00000100:00100000:1.0:1789762907.192177:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380435785812 time 380435888207 state 0x2 ref 0 00000100:00100000:3.0:1789762907.194393:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762907.194473:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067504128 00000100:00100000:2.0:1789762907.194482:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067504128 00000100:00100000:2.0:1789762907.194517:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380435785812 time 380435888207 state 0x0 ref 1 00000100:00100000:2.0:1789762907.194528:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380535888207 time 380435888207 state 0x1 ref 1 00000100:00100000:1.0:1789762907.288942:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380535888207 time 380535983148 state 0x0 ref 1 00000100:00100000:1.0:1789762907.288970:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067504128:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762907.289007:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17702b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.289020:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762907.289024:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762907.289029:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17702b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.289040:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17702b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.291960:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17702b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.291971:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17702b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.291978:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17702b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.291982:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762907.291986:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17702b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.291992:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17702b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.292005:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17702b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.292009:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17702b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.292073:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067504128:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3115us (97680us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762907.292081:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 863 00000100:00100000:1.0:1789762907.292102:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380535888207 time 380535983148 state 0x2 ref 0 00000100:00100000:3.0:1789762907.294278:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762907.294324:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067504256 00000100:00100000:1.0:1789762907.294331:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067504256 00000100:00100000:1.0:1789762907.294366:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380535888207 time 380535983148 state 0x0 ref 1 00000100:00100000:1.0:1789762907.294376:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380635983148 time 380535983148 state 0x1 ref 1 00000100:00100000:3.0:1789762907.389073:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380635983148 time 380636113578 state 0x0 ref 1 00000100:00100000:3.0:1789762907.389110:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067504256:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762907.389181:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f177032 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.389200:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762907.389206:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762907.389213:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f177032 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.389229:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f177032 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.392766:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f177032 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.392781:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f177032 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.392793:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f177032 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.392800:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f0c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762907.392803:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f177032 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.392810:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f177032 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.392825:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f177032 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.392830:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003f1b8f8b/0xdf5a7ac59f177032 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762907.392883:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067504256:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3787us (98605us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762907.392892:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 864 00000100:00100000:3.0:1789762907.392902:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380635983148 time 380636113578 state 0x2 ref 0 00000100:00100000:3.0:1789762907.395071:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762907.395124:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067504384 00000100:00100000:1.0:1789762907.395133:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067504384 00000100:00100000:1.0:1789762907.395183:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380635983148 time 380636113578 state 0x0 ref 1 00000100:00100000:1.0:1789762907.395196:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380736113578 time 380636113578 state 0x1 ref 1 00000100:00100000:0.0:1789762907.489133:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380736113578 time 380736199478 state 0x0 ref 1 00000100:00100000:0.0:1789762907.489154:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067504384:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762907.489187:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177039 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.489201:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762907.489205:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762907.489210:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177039 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.489221:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177039 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.491752:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177039 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.491763:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177039 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.491770:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177039 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.491775:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762907.491778:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177039 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.491784:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177039 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.491797:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177039 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.491803:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177039 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762907.491859:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067504384:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2712us (96788us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762907.491866:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 865 00000100:00100000:0.0:1789762907.491875:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380736113578 time 380736199478 state 0x2 ref 0 00000100:00100000:0.0:1789762907.493934:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762907.493978:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067504512 00000100:00100000:1.0:1789762907.493987:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067504512 00000100:00100000:1.0:1789762907.494031:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380736113578 time 380736199478 state 0x0 ref 1 00000100:00100000:1.0:1789762907.494054:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380836199478 time 380736199478 state 0x1 ref 1 00000100:00100000:0.0:1789762907.589207:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380836199478 time 380836273647 state 0x0 ref 1 00000100:00100000:0.0:1789762907.589230:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067504512:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762907.589263:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177040 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.589277:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762907.589280:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762907.589285:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177040 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.589296:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177040 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.592436:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177040 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.592447:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177040 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.592454:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177040 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.592463:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762907.592468:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177040 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.592476:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177040 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.592496:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177040 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.592506:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177040 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762907.592559:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067504512:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3337us (98640us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762907.592567:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 866 00000100:00100000:0.0:1789762907.592576:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380836199478 time 380836273647 state 0x2 ref 0 00000100:00100000:0.0:1789762907.594539:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762907.594582:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067504640 00000100:00100000:3.0:1789762907.594590:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067504640 00000100:00100000:3.0:1789762907.594626:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380836199478 time 380836273647 state 0x0 ref 1 00000100:00100000:3.0:1789762907.594637:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380936273647 time 380836273647 state 0x1 ref 1 00000100:00100000:1.0:1789762907.648055:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789762907.648078:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8aa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074727424:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762907.648138:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762907.648183:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074727424 00000100:00100000:0.0:1789762907.648189:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074727424 00000100:00100000:0.0:1789762907.648235:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:0.0:1789762907.648245:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074727424:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762907.648280:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762907.648328:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074727424:12345-0@lo:41:osp-pre-1-0.0 Request processed in 84us (192us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762907.648335:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:1.0:1789762907.648367:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8aa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074727424:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762907.689307:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380936273647 time 380936374065 state 0x0 ref 1 00000100:00100000:2.0:1789762907.689330:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067504640:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762907.689377:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f177047 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.689391:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762907.689394:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762907.689398:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f177047 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.689409:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f177047 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.691940:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f177047 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.691954:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f177047 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.691964:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f177047 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.691972:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5933c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762907.691977:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f177047 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.691987:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f177047 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.692014:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f177047 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.692020:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd7b70f4/0xdf5a7ac59f177047 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762907.692076:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067504640:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2754us (97537us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762907.692083:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 867 00000100:00100000:2.0:1789762907.692091:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380936273647 time 380936374065 state 0x2 ref 0 00000100:00100000:3.0:1789762907.693821:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762907.693860:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067504768 00000100:00100000:0.0:1789762907.693868:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067504768 00000100:00100000:0.0:1789762907.693904:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 380936273647 time 380936374065 state 0x0 ref 1 00000100:00100000:0.0:1789762907.693915:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381036374065 time 380936374065 state 0x1 ref 1 00010000:00010000:2.0:1789762907.775989:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762907.789365:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381036374065 time 381036433190 state 0x0 ref 1 00000100:00100000:0.0:1789762907.789387:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067504768:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762907.789421:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17704e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.789435:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762907.789438:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762907.789442:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17704e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.789452:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17704e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.792172:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17704e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.792189:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17704e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.792200:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17704e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.792210:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762907.792215:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17704e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.792226:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17704e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.792244:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17704e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.792252:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17704e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.792319:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067504768:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2937us (98495us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762907.792335:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 868 00000100:00100000:1.0:1789762907.792347:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381036374065 time 381036433190 state 0x2 ref 0 00000100:00100000:3.0:1789762907.794442:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762907.794492:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067504896 00000100:00100000:0.0:1789762907.794500:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067504896 00000100:00100000:0.0:1789762907.794536:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381036374065 time 381036433190 state 0x0 ref 1 00000100:00100000:0.0:1789762907.794547:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381136433190 time 381036433190 state 0x1 ref 1 00000100:00100000:1.0:1789762907.889487:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381136433190 time 381136552902 state 0x0 ref 1 00000100:00100000:1.0:1789762907.889511:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff69aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067504896:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762907.889546:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177055 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.889560:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762907.889563:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762907.889568:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177055 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.889580:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177055 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.892402:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177055 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.892413:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177055 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.892419:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177055 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.892428:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762907.892431:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177055 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.892437:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177055 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.892450:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177055 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.892455:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177055 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.892510:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff69aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067504896:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3008us (98070us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762907.892517:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 869 00000100:00100000:1.0:1789762907.892525:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381136433190 time 381136552902 state 0x2 ref 0 00000100:00100000:2.0:1789762907.894355:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762907.894394:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067505024 00000100:00100000:1.0:1789762907.894402:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067505024 00000100:00100000:1.0:1789762907.894435:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381136433190 time 381136552902 state 0x0 ref 1 00000100:00100000:1.0:1789762907.894445:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381236552902 time 381136552902 state 0x1 ref 1 00000100:00100000:1.0:1789762907.989554:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381236552902 time 381236619665 state 0x0 ref 1 00000100:00100000:1.0:1789762907.989581:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067505024:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762907.989615:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17705c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.989628:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762907.989632:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762907.989636:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17705c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.989647:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17705c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.992395:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17705c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.992410:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17705c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.992421:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17705c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.992430:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762907.992436:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17705c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.992447:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17705c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.992468:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17705c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.992485:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17705c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.992546:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067505024:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2967us (98185us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762907.992555:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 870 00000100:00100000:1.0:1789762907.992563:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381236552902 time 381236619665 state 0x2 ref 0 00000100:00100000:2.0:1789762907.994387:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762907.994423:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067505152 00000100:00100000:1.0:1789762907.994430:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067505152 00000100:00100000:1.0:1789762907.994467:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381236552902 time 381236619665 state 0x0 ref 1 00000100:00100000:1.0:1789762907.994477:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381336619665 time 381236619665 state 0x1 ref 1 00000100:00100000:3.0:1789762908.089669:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381336619665 time 381336733892 state 0x0 ref 1 00000100:00100000:3.0:1789762908.089694:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067505152:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762908.089729:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177063 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.089743:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762908.089747:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762908.089751:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177063 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.089764:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177063 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.093499:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177063 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.093514:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177063 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.093524:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177063 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.093533:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762908.093538:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177063 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.093547:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177063 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.093564:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177063 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.093572:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177063 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762908.093633:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067505152:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3947us (99245us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762908.093642:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 871 00000100:00100000:3.0:1789762908.093653:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381336619665 time 381336733892 state 0x2 ref 0 00000100:00100000:2.0:1789762908.096470:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762908.096584:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067505280 00000100:00100000:1.0:1789762908.096592:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067505280 00000100:00100000:1.0:1789762908.096642:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381336619665 time 381336733892 state 0x0 ref 1 00000100:00100000:1.0:1789762908.096653:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381436733892 time 381336733892 state 0x1 ref 1 00000100:00100000:0.0:1789762908.189751:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381436733892 time 381436812280 state 0x0 ref 1 00000100:00100000:0.0:1789762908.189779:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9246d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067505280:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762908.189818:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000080e06ec4/0xdf5a7ac59f17706a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.189832:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762908.189835:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762908.189840:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080e06ec4/0xdf5a7ac59f17706a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.189851:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080e06ec4/0xdf5a7ac59f17706a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.192731:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000080e06ec4/0xdf5a7ac59f17706a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.192743:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080e06ec4/0xdf5a7ac59f17706a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.192751:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080e06ec4/0xdf5a7ac59f17706a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.192756:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a6800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762908.192767:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080e06ec4/0xdf5a7ac59f17706a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.192775:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080e06ec4/0xdf5a7ac59f17706a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.192791:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080e06ec4/0xdf5a7ac59f17706a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.192796:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080e06ec4/0xdf5a7ac59f17706a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762908.192845:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9246d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067505280:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3078us (96381us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762908.192853:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 872 00000100:00100000:2.0:1789762908.192861:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381436733892 time 381436812280 state 0x2 ref 0 00000100:00100000:2.0:1789762908.194758:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762908.194802:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067505408 00000100:00100000:1.0:1789762908.194810:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067505408 00000100:00100000:1.0:1789762908.194845:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381436733892 time 381436812280 state 0x0 ref 1 00000100:00100000:1.0:1789762908.194855:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381536812280 time 381436812280 state 0x1 ref 1 00000100:00100000:0.0:1789762908.289837:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381536812280 time 381536903670 state 0x0 ref 1 00000100:00100000:0.0:1789762908.289862:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9245f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067505408:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762908.289897:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f177071 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.289910:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762908.289913:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762908.289945:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f177071 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.289963:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f177071 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.292554:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f177071 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.292567:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f177071 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.292574:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f177071 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.292579:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a7c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762908.292583:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f177071 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.292589:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f177071 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.292602:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f177071 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.292608:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f177071 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762908.292662:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9245f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067505408:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2809us (97903us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762908.292670:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 873 00000100:00100000:1.0:1789762908.292678:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381536812280 time 381536903670 state 0x2 ref 0 00000100:00100000:0.0:1789762908.294522:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762908.294562:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067505536 00000100:00100000:3.0:1789762908.294571:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067505536 00000100:00100000:3.0:1789762908.294609:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381536812280 time 381536903670 state 0x0 ref 1 00000100:00100000:3.0:1789762908.294619:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381636903670 time 381536903670 state 0x1 ref 1 00000100:00100000:3.0:1789762908.389897:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381636903670 time 381636963159 state 0x0 ref 1 00000100:00100000:3.0:1789762908.389940:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067505536:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762908.389977:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177078 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.389991:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762908.389995:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762908.389999:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177078 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.390009:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177078 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.393241:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177078 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.393253:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177078 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.393261:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177078 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.393267:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762908.393271:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177078 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.393277:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177078 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.393292:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177078 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.393298:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177078 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762908.393348:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067505536:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3435us (98825us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762908.393356:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 874 00000100:00100000:3.0:1789762908.393365:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381636903670 time 381636963159 state 0x2 ref 0 00000100:00100000:0.0:1789762908.395283:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762908.395345:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067505664 00000100:00100000:2.0:1789762908.395353:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067505664 00000100:00100000:2.0:1789762908.395391:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381636903670 time 381636963159 state 0x0 ref 1 00000100:00100000:2.0:1789762908.395402:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381736963159 time 381636963159 state 0x1 ref 1 00000100:00100000:0.0:1789762908.490013:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381736963159 time 381737079200 state 0x0 ref 1 00000100:00100000:0.0:1789762908.490055:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067505664:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762908.490097:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033e9a6f8/0xdf5a7ac59f17707f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.490116:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762908.490121:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762908.490128:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033e9a6f8/0xdf5a7ac59f17707f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.490141:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033e9a6f8/0xdf5a7ac59f17707f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.492996:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033e9a6f8/0xdf5a7ac59f17707f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.493007:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033e9a6f8/0xdf5a7ac59f17707f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.493014:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033e9a6f8/0xdf5a7ac59f17707f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.493020:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a6c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762908.493024:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033e9a6f8/0xdf5a7ac59f17707f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.493030:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033e9a6f8/0xdf5a7ac59f17707f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.493057:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033e9a6f8/0xdf5a7ac59f17707f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.493063:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033e9a6f8/0xdf5a7ac59f17707f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762908.493117:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067505664:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3087us (97834us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762908.493124:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 875 00000100:00100000:1.0:1789762908.493132:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381736963159 time 381737079200 state 0x2 ref 0 00000100:00100000:2.0:1789762908.494992:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762908.495026:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067505792 00000100:00100000:1.0:1789762908.495034:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067505792 00000100:00100000:1.0:1789762908.495084:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381736963159 time 381737079200 state 0x0 ref 1 00000100:00100000:1.0:1789762908.495096:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381837079200 time 381737079200 state 0x1 ref 1 00000100:00100000:1.0:1789762908.590102:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381837079200 time 381837168779 state 0x0 ref 1 00000100:00100000:1.0:1789762908.590126:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9244e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067505792:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762908.590160:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177086 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.590173:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762908.590176:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762908.590181:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177086 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.590192:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177086 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.593108:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177086 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.593120:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177086 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.593128:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177086 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.593134:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762908.593139:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177086 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.593146:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177086 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.593160:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177086 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.593166:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f177086 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762908.593217:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9244e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067505792:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3100us (98224us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762908.593224:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 876 00000100:00100000:1.0:1789762908.593233:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381837079200 time 381837168779 state 0x2 ref 0 00000100:00100000:2.0:1789762908.595118:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762908.595168:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067505920 00000100:00100000:3.0:1789762908.595182:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067505920 00000100:00100000:3.0:1789762908.595240:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381837079200 time 381837168779 state 0x0 ref 1 00000100:00100000:3.0:1789762908.595254:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381937168779 time 381837168779 state 0x1 ref 1 00000100:00100000:1.0:1789762908.672124:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c94a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074727552:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762908.672185:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762908.672244:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074727552 00000100:00100000:3.0:1789762908.672253:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074727552 00000100:00100000:3.0:1789762908.672330:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:3.0:1789762908.672344:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8aa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074727552:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789762908.672386:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:3.0:1789762908.672461:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8aa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074727552:12345-0@lo:41:osp-pre-0-1.0 Request processed in 123us (280us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762908.672467:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00100000:1.0:1789762908.672521:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c94a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074727552:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762908.690185:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381937168779 time 381937256217 state 0x0 ref 1 00000100:00100000:1.0:1789762908.690203:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067505920:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762908.690232:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17708d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.690245:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762908.690248:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762908.690253:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17708d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.690263:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17708d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.692828:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17708d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.692839:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17708d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.692846:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17708d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.692851:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762908.692854:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17708d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.692859:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17708d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.692871:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17708d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.692876:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f17708d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762908.692953:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067505920:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2755us (97836us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762908.692960:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 877 00000100:00100000:1.0:1789762908.692969:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381937168779 time 381937256217 state 0x2 ref 0 00000100:00100000:2.0:1789762908.694714:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762908.694804:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067506048 00000100:00100000:1.0:1789762908.694812:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067506048 00000100:00100000:1.0:1789762908.694846:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 381937168779 time 381937256217 state 0x0 ref 1 00000100:00100000:1.0:1789762908.694857:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382037256217 time 381937256217 state 0x1 ref 1 00000100:00100000:3.0:1789762908.790306:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382037256217 time 382037364584 state 0x0 ref 1 00000100:00100000:3.0:1789762908.790338:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067506048:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762908.790386:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000098b10c6a/0xdf5a7ac59f177094 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.790408:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762908.790414:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762908.790422:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098b10c6a/0xdf5a7ac59f177094 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762908.790445:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098b10c6a/0xdf5a7ac59f177094 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.794583:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000098b10c6a/0xdf5a7ac59f177094 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.794596:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098b10c6a/0xdf5a7ac59f177094 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.794608:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098b10c6a/0xdf5a7ac59f177094 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.794615:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762908.794620:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098b10c6a/0xdf5a7ac59f177094 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.794627:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098b10c6a/0xdf5a7ac59f177094 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.794644:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098b10c6a/0xdf5a7ac59f177094 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.794649:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098b10c6a/0xdf5a7ac59f177094 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762908.794719:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067506048:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4393us (100005us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762908.794727:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 878 00000100:00100000:0.0:1789762908.794736:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382037256217 time 382037364584 state 0x2 ref 0 00000100:00100000:3.0:1789762908.796521:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762908.796613:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067506176 00000100:00100000:1.0:1789762908.796622:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067506176 00000100:00100000:1.0:1789762908.796671:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382037256217 time 382037364584 state 0x0 ref 1 00000100:00100000:1.0:1789762908.796683:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382137364584 time 382037364584 state 0x1 ref 1 00010000:00010000:3.0:1789762908.801056:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762908.890390:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382137364584 time 382137458415 state 0x0 ref 1 00000100:00100000:2.0:1789762908.890409:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff699f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067506176:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762908.890441:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17709b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.890456:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762908.890459:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762908.890465:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17709b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.890481:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17709b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.893308:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17709b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.893317:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17709b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.893323:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17709b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.893327:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762908.893331:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17709b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.893354:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17709b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.893367:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17709b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762908.893372:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17709b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762908.893420:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff699f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067506176:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3017us (96907us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762908.893426:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 879 00000100:00100000:2.0:1789762908.893433:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382137364584 time 382137458415 state 0x2 ref 0 00000100:00100000:2.0:1789762908.895314:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762908.895365:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067506304 00000100:00100000:1.0:1789762908.895372:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067506304 00000100:00100000:1.0:1789762908.895409:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382137364584 time 382137458415 state 0x0 ref 1 00000100:00100000:1.0:1789762908.895419:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382237458415 time 382137458415 state 0x1 ref 1 00000100:00100000:0.0:1789762908.990500:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382237458415 time 382237566262 state 0x0 ref 1 00000100:00100000:0.0:1789762908.990535:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067506304:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762908.990578:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f1770a2 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.990593:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762908.990597:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762908.990602:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f1770a2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762908.990614:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f1770a2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.994130:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f1770a2 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.994143:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f1770a2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.994150:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f1770a2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.994157:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762908.994161:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f1770a2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.994167:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f1770a2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.994186:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f1770a2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762908.994192:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea4f5489/0xdf5a7ac59f1770a2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762908.994254:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067506304:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3729us (98939us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762908.994264:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 880 00000100:00100000:1.0:1789762908.994273:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382237458415 time 382237566262 state 0x2 ref 0 00000100:00100000:2.0:1789762908.996273:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762908.996323:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067506432 00000100:00100000:0.0:1789762908.996332:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067506432 00000100:00100000:0.0:1789762908.996370:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382237458415 time 382237566262 state 0x0 ref 1 00000100:00100000:0.0:1789762908.996381:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382337566262 time 382237566262 state 0x1 ref 1 00000100:00100000:3.0:1789762909.090636:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382337566262 time 382337696804 state 0x0 ref 1 00000100:00100000:3.0:1789762909.090685:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067506432:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:1.0:1789762909.090717:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762909.090731:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770a9 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.090749:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762909.090754:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762909.090760:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770a9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.090777:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770a9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.112802:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770a9 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.112829:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770a9 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.112840:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770a9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.112848:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762909.112854:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770a9 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.112864:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770a9 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.112888:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770a9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.112897:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770a9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762909.113082:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067506432:12345-192.168.203.37@tcp:3:dd.500 Request processed in 22417us (116798us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762909.113099:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 881 00000100:00100000:0.0:1789762909.113112:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382337566262 time 382337696804 state 0x2 ref 0 00000100:00100000:2.0:1789762909.115156:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762909.115211:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067506560 00000100:00100000:0.0:1789762909.115220:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067506560 00000100:00100000:0.0:1789762909.115261:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382337566262 time 382337696804 state 0x0 ref 1 00000100:00100000:0.0:1789762909.115275:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382437696804 time 382337696804 state 0x1 ref 1 00000100:00100000:0.0:1789762909.190664:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382437696804 time 382437733147 state 0x0 ref 1 00000100:00100000:0.0:1789762909.190683:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067506560:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762909.190717:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000093ef9fa9/0xdf5a7ac59f1770b0 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.190732:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762909.190736:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762909.190740:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093ef9fa9/0xdf5a7ac59f1770b0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.190751:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093ef9fa9/0xdf5a7ac59f1770b0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.193618:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000093ef9fa9/0xdf5a7ac59f1770b0 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.193630:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093ef9fa9/0xdf5a7ac59f1770b0 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.193637:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093ef9fa9/0xdf5a7ac59f1770b0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.193648:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762909.193652:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093ef9fa9/0xdf5a7ac59f1770b0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.193658:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093ef9fa9/0xdf5a7ac59f1770b0 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.193671:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093ef9fa9/0xdf5a7ac59f1770b0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.193676:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093ef9fa9/0xdf5a7ac59f1770b0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762909.193727:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067506560:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3048us (78571us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762909.193735:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 882 00000100:00100000:0.0:1789762909.193744:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382437696804 time 382437733147 state 0x2 ref 0 00000100:00100000:0.0:1789762909.195596:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762909.195649:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067506688 00000100:00100000:1.0:1789762909.195657:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067506688 00000100:00100000:1.0:1789762909.195693:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382437696804 time 382437733147 state 0x0 ref 1 00000100:00100000:1.0:1789762909.195703:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382537733147 time 382437733147 state 0x1 ref 1 00000100:00100000:1.0:1789762909.290724:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382537733147 time 382537792285 state 0x0 ref 1 00000100:00100000:1.0:1789762909.290745:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067506688:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762909.290777:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770b7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.290795:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762909.290798:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762909.290802:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770b7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.290812:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770b7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.293403:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770b7 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.293417:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770b7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.293427:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770b7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.293435:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762909.293441:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770b7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.293451:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770b7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.293469:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770b7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.293479:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770b7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762909.293527:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067506688:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2788us (97931us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762909.293537:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 883 00000100:00100000:1.0:1789762909.293548:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382537733147 time 382537792285 state 0x2 ref 0 00000100:00100000:2.0:1789762909.295337:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762909.295376:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067506816 00000100:00100000:0.0:1789762909.295383:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067506816 00000100:00100000:0.0:1789762909.295416:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382537733147 time 382537792285 state 0x0 ref 1 00000100:00100000:0.0:1789762909.295426:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382637792285 time 382537792285 state 0x1 ref 1 00000100:00100000:1.0:1789762909.390835:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382637792285 time 382637902739 state 0x0 ref 1 00000100:00100000:1.0:1789762909.390859:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067506816:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762909.390892:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770be lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.390911:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762909.390914:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762909.390939:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770be lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.390950:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770be lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.393553:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770be lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.393576:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770be lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.393586:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770be lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.393594:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762909.393597:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770be lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.393604:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770be lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.393618:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770be lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.393624:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fdbd7cc/0xdf5a7ac59f1770be lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762909.393673:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067506816:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2824us (98336us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762909.393681:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 884 00000100:00100000:1.0:1789762909.393688:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382637792285 time 382637902739 state 0x2 ref 0 00000100:00100000:2.0:1789762909.395480:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762909.395529:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067506944 00000100:00100000:0.0:1789762909.395538:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067506944 00000100:00100000:0.0:1789762909.395576:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382637792285 time 382637902739 state 0x0 ref 1 00000100:00100000:0.0:1789762909.395586:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382737902739 time 382637902739 state 0x1 ref 1 00000100:00100000:0.0:1789762909.490876:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382737902739 time 382737945409 state 0x0 ref 1 00000100:00100000:0.0:1789762909.490895:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067506944:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762909.490940:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d09fbdd8/0xdf5a7ac59f1770c5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.490953:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762909.490957:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762909.490961:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d09fbdd8/0xdf5a7ac59f1770c5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.490971:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d09fbdd8/0xdf5a7ac59f1770c5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.493520:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d09fbdd8/0xdf5a7ac59f1770c5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.493533:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d09fbdd8/0xdf5a7ac59f1770c5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.493547:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d09fbdd8/0xdf5a7ac59f1770c5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.493552:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d98e6000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762909.493555:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d09fbdd8/0xdf5a7ac59f1770c5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.493561:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d09fbdd8/0xdf5a7ac59f1770c5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.493573:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d09fbdd8/0xdf5a7ac59f1770c5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.493579:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d09fbdd8/0xdf5a7ac59f1770c5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762909.493622:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067506944:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2733us (98143us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762909.493629:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 885 00000100:00100000:0.0:1789762909.493636:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382737902739 time 382737945409 state 0x2 ref 0 00000100:00100000:0.0:1789762909.495327:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762909.495372:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067507072 00000100:00100000:1.0:1789762909.495378:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067507072 00000100:00100000:1.0:1789762909.495411:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382737902739 time 382737945409 state 0x0 ref 1 00000100:00100000:1.0:1789762909.495420:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382837945409 time 382737945409 state 0x1 ref 1 00000100:00100000:2.0:1789762909.590999:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382837945409 time 382838066372 state 0x0 ref 1 00000100:00100000:2.0:1789762909.591024:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff883800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067507072:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762909.591074:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a6f7bbef/0xdf5a7ac59f1770cc lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.591109:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762909.591114:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762909.591122:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6f7bbef/0xdf5a7ac59f1770cc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.591137:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6f7bbef/0xdf5a7ac59f1770cc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.593980:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a6f7bbef/0xdf5a7ac59f1770cc lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.593992:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6f7bbef/0xdf5a7ac59f1770cc lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.593999:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6f7bbef/0xdf5a7ac59f1770cc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.594005:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8c83800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762909.594009:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6f7bbef/0xdf5a7ac59f1770cc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.594016:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6f7bbef/0xdf5a7ac59f1770cc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.594043:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6f7bbef/0xdf5a7ac59f1770cc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.594060:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6f7bbef/0xdf5a7ac59f1770cc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762909.594106:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff883800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067507072:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3092us (98780us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762909.594113:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 886 00000100:00100000:2.0:1789762909.594121:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382837945409 time 382838066372 state 0x2 ref 0 00000100:00100000:2.0:1789762909.595880:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762909.595945:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067507200 00000100:00100000:1.0:1789762909.595952:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067507200 00000100:00100000:1.0:1789762909.595989:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382837945409 time 382838066372 state 0x0 ref 1 00000100:00100000:1.0:1789762909.596000:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382938066372 time 382838066372 state 0x1 ref 1 00000100:00100000:0.0:1789762909.639672:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762909.639722:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067507328 00000100:00100000:2.0:1789762909.639728:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067507328 00000100:00100000:0.0:1789762909.639739:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762909.639765:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762909.639775:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 375 00000100:00100000:2.0:1789762909.639788:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb938850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067507328:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762909.639790:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067507456 00000100:00100000:0.0:1789762909.639796:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762909.639798:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067507456 00000100:00100000:1.0:1789762909.639820:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067507712 00000100:00100000:1.0:1789762909.639826:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067507712 00000100:00100000:0.0:1789762909.639835:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762909.639845:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb938850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067507328:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 61us (177us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762909.639851:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 375 00000100:00100000:3.0:1789762909.639866:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067507584 00000100:00100000:3.0:1789762909.639871:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067507584 00000100:00100000:1.0:1789762909.639875:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 234 00000100:00100000:1.0:1789762909.639885:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff883b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067507712:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762909.639902:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067507840 00000100:00100000:2.0:1789762909.639906:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067507840 00000100:00100000:3.0:1789762909.639909:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 568 00000100:00100000:1.0:1789762909.639958:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff883b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067507712:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 73us (162us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762909.639962:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762909.639969:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 569 00000100:00100000:1.0:1789762909.639977:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 234 00000100:00100000:2.0:1789762909.639982:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff883100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067507584:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762909.640011:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff883100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067507584:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 34us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762909.640013:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067507968 00000100:00100000:2.0:1789762909.640021:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 569 00000100:00100000:1.0:1789762909.640022:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067507968 00000100:00100000:3.0:1789762909.640022:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2398:x1876702067507456:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762909.640061:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 570 00000100:00100000:0.0:1789762909.640075:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff883480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067507840:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762909.640077:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 571 00000100:00100000:2.0:1789762909.640101:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067507968:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762909.640124:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff883480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067507840:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 53us (289us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762909.640131:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 570 00000100:00100000:2.0:1789762909.640152:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067507968:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 54us (192us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762909.640159:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 571 00000100:00080000:0.0:1789762909.640172:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762909.640178:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762909.640180:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762909.640201:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2398:x1876702067507456:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 226us (403us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762909.640210:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 568 00000100:00100000:1.0:1789762909.691070:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382938066372 time 382938132384 state 0x0 ref 1 00000100:00100000:1.0:1789762909.691092:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067507200:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762909.691122:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000592e99d6/0xdf5a7ac59f1770d3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.691140:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762909.691143:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762909.691148:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000592e99d6/0xdf5a7ac59f1770d3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.691158:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000592e99d6/0xdf5a7ac59f1770d3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.693724:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000592e99d6/0xdf5a7ac59f1770d3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.693735:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000592e99d6/0xdf5a7ac59f1770d3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.693742:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000592e99d6/0xdf5a7ac59f1770d3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.693749:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a4200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762909.693753:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000592e99d6/0xdf5a7ac59f1770d3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.693763:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000592e99d6/0xdf5a7ac59f1770d3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.693787:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000592e99d6/0xdf5a7ac59f1770d3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.693797:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000592e99d6/0xdf5a7ac59f1770d3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762909.693844:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067507200:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2762us (97963us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762909.693852:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 887 00000100:00100000:0.0:1789762909.693859:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382938066372 time 382938132384 state 0x2 ref 0 00000100:00100000:0.0:1789762909.695666:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762909.695714:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067508096 00000100:00100000:3.0:1789762909.695720:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067508096 00000100:00100000:3.0:1789762909.695752:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 382938066372 time 382938132384 state 0x0 ref 1 00000100:00100000:3.0:1789762909.695762:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383038132384 time 382938132384 state 0x1 ref 1 00000100:00100000:0.0:1789762909.791206:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383038132384 time 383038271477 state 0x0 ref 1 00000100:00100000:0.0:1789762909.791229:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067508096:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762909.791266:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f1770da lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.791281:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762909.791284:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762909.791289:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f1770da lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.791302:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f1770da lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.795807:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f1770da lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.795823:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f1770da lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.795833:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f1770da lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.795842:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762909.795848:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f1770da lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.795858:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f1770da lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.795880:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f1770da lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.795897:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081139dbe/0xdf5a7ac59f1770da lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762909.796007:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067508096:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4784us (100338us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762909.796018:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 888 00000100:00100000:1.0:1789762909.796044:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383038132384 time 383038271477 state 0x2 ref 0 00000100:00100000:2.0:1789762909.798235:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762909.798280:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067508224 00000100:00100000:0.0:1789762909.798288:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067508224 00000100:00100000:0.0:1789762909.798334:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383038132384 time 383038271477 state 0x0 ref 1 00000100:00100000:0.0:1789762909.798345:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383138271477 time 383038271477 state 0x1 ref 1 00010000:00010000:0.0:1789762909.824078:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762909.891266:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383138271477 time 383138335327 state 0x0 ref 1 00000100:00100000:0.0:1789762909.891293:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067508224:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762909.891349:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1770e1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.891362:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762909.891370:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762909.891375:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1770e1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.891385:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1770e1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.895101:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1770e1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.895115:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1770e1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.895125:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1770e1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.895134:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762909.895140:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1770e1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.895150:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1770e1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.895169:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1770e1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.895179:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c4dc1f5/0xdf5a7ac59f1770e1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762909.895236:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067508224:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3950us (96999us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762909.895247:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 889 00000100:00100000:1.0:1789762909.895258:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383138271477 time 383138335327 state 0x2 ref 0 00000100:00100000:2.0:1789762909.897257:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762909.897314:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067508352 00000100:00100000:0.0:1789762909.897321:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067508352 00000100:00100000:0.0:1789762909.897365:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383138271477 time 383138335327 state 0x0 ref 1 00000100:00100000:0.0:1789762909.897378:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383238335327 time 383138335327 state 0x1 ref 1 00000100:00100000:1.0:1789762909.991379:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383238335327 time 383238442901 state 0x0 ref 1 00000100:00100000:1.0:1789762909.991406:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067508352:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762909.991446:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009fc1d413/0xdf5a7ac59f1770e8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.991463:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762909.991468:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762909.991475:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fc1d413/0xdf5a7ac59f1770e8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.991489:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fc1d413/0xdf5a7ac59f1770e8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.995295:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009fc1d413/0xdf5a7ac59f1770e8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.995312:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009fc1d413/0xdf5a7ac59f1770e8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.995324:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009fc1d413/0xdf5a7ac59f1770e8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.995333:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d98e7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762909.995339:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009fc1d413/0xdf5a7ac59f1770e8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.995350:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009fc1d413/0xdf5a7ac59f1770e8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.995367:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009fc1d413/0xdf5a7ac59f1770e8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.995373:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fc1d413/0xdf5a7ac59f1770e8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762909.995437:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067508352:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4040us (98178us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762909.995448:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 890 00000100:00100000:0.0:1789762909.995456:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383238335327 time 383238442901 state 0x2 ref 0 00000100:00100000:2.0:1789762909.998355:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762909.998413:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067508480 00000100:00100000:1.0:1789762909.998422:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067508480 00000100:00100000:1.0:1789762909.998464:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383238335327 time 383238442901 state 0x0 ref 1 00000100:00100000:1.0:1789762909.998479:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383338442901 time 383238442901 state 0x1 ref 1 00000100:00100000:1.0:1789762910.091467:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383338442901 time 383338527729 state 0x0 ref 1 00000100:00100000:1.0:1789762910.091504:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067508480:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762910.091558:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003ebf48be/0xdf5a7ac59f1770ef lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.091575:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762910.091580:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762910.091586:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ebf48be/0xdf5a7ac59f1770ef lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.091600:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ebf48be/0xdf5a7ac59f1770ef lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.096146:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003ebf48be/0xdf5a7ac59f1770ef lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.096190:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ebf48be/0xdf5a7ac59f1770ef lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.096208:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ebf48be/0xdf5a7ac59f1770ef lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.096223:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d98e7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762910.096233:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ebf48be/0xdf5a7ac59f1770ef lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.096250:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ebf48be/0xdf5a7ac59f1770ef lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.096285:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ebf48be/0xdf5a7ac59f1770ef lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.096298:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ebf48be/0xdf5a7ac59f1770ef lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762910.096437:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067508480:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4942us (98073us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762910.096454:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 891 00000100:00100000:3.0:1789762910.096471:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383338442901 time 383338527729 state 0x2 ref 0 00000100:00100000:2.0:1789762910.098983:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762910.099050:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067508608 00000100:00100000:1.0:1789762910.099059:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067508608 00000100:00100000:1.0:1789762910.099121:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383338442901 time 383338527729 state 0x0 ref 1 00000100:00100000:1.0:1789762910.099138:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383438527729 time 383338527729 state 0x1 ref 1 00000100:00100000:3.0:1789762910.191600:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383438527729 time 383438664134 state 0x0 ref 1 00000100:00100000:3.0:1789762910.191635:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c94a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067508608:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762910.191675:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770f6 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.191689:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762910.191693:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762910.191698:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770f6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.191714:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770f6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.195615:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770f6 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.195631:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770f6 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.195642:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770f6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.195651:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762910.195657:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770f6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.195668:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770f6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.195696:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770f6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.195706:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1770f6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762910.195777:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c94a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067508608:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4159us (96795us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762910.195788:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 892 00000100:00100000:3.0:1789762910.195800:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383438527729 time 383438664134 state 0x2 ref 0 00000100:00100000:2.0:1789762910.199548:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762910.199595:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067508736 00000100:00100000:0.0:1789762910.199605:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067508736 00000100:00100000:0.0:1789762910.199659:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383438527729 time 383438664134 state 0x0 ref 1 00000100:00100000:0.0:1789762910.199676:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383538664134 time 383438664134 state 0x1 ref 1 00000100:00100000:0.0:1789762910.291710:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383538664134 time 383538770611 state 0x0 ref 1 00000100:00100000:0.0:1789762910.291741:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067508736:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762910.291780:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f1770fd lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.291794:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762910.291798:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762910.291803:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f1770fd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.291814:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f1770fd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.294751:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f1770fd lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.294773:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f1770fd lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.294783:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f1770fd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.294791:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762910.294796:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f1770fd lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.294807:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f1770fd lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.294826:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f1770fd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.294835:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f1770fd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762910.294889:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067508736:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3162us (95344us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762910.294899:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 893 00000100:00100000:0.0:1789762910.294910:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383538664134 time 383538770611 state 0x2 ref 0 00000100:00100000:2.0:1789762910.297590:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762910.297641:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067508864 00000100:00100000:3.0:1789762910.297651:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067508864 00000100:00100000:3.0:1789762910.297708:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383538664134 time 383538770611 state 0x0 ref 1 00000100:00100000:3.0:1789762910.297722:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383638770611 time 383538770611 state 0x1 ref 1 00000100:00100000:0.0:1789762910.391801:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383638770611 time 383638865547 state 0x0 ref 1 00000100:00100000:0.0:1789762910.391832:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8eddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067508864:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762910.391869:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177104 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.391883:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762910.391887:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762910.391891:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177104 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.391902:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177104 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.396068:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177104 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.396079:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177104 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.396087:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177104 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.396093:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762910.396097:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177104 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.396103:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177104 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.396118:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177104 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.396123:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177104 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762910.396171:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8eddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067508864:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4353us (98583us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762910.396179:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 894 00000100:00100000:0.0:1789762910.396188:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383638770611 time 383638865547 state 0x2 ref 0 00000100:00100000:2.0:1789762910.398699:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762910.398746:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067508992 00000100:00100000:3.0:1789762910.398756:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067508992 00000100:00100000:3.0:1789762910.398799:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383638770611 time 383638865547 state 0x0 ref 1 00000100:00100000:3.0:1789762910.398815:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383738865547 time 383638865547 state 0x1 ref 1 00000100:00100000:1.0:1789762910.491984:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383738865547 time 383739042334 state 0x0 ref 1 00000100:00100000:1.0:1789762910.492016:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8ede300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067508992:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762910.492086:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f17710b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.492105:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762910.492110:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762910.492117:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f17710b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.492132:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f17710b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.495263:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f17710b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.495278:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f17710b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.495290:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f17710b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.495299:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762910.495303:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f17710b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.495314:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f17710b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.495333:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f17710b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.495342:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f17710b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762910.495402:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8ede300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067508992:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3397us (96702us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762910.495414:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 895 00000100:00100000:1.0:1789762910.495426:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383738865547 time 383739042334 state 0x2 ref 0 00000100:00100000:1.0:1789762910.497803:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762910.497851:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067509120 00000100:00100000:0.0:1789762910.497860:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067509120 00000100:00100000:0.0:1789762910.497900:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383738865547 time 383739042334 state 0x0 ref 1 00000100:00100000:0.0:1789762910.497911:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383839042334 time 383739042334 state 0x1 ref 1 00000100:00100000:1.0:1789762910.592123:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383839042334 time 383839177214 state 0x0 ref 1 00000100:00100000:1.0:1789762910.592155:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067509120:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762910.592193:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177112 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.592208:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762910.592211:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762910.592216:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177112 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.592229:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177112 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.595402:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177112 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.595419:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177112 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.595431:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177112 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.595447:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762910.595454:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177112 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.595464:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177112 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.595481:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177112 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.595487:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177112 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762910.595536:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067509120:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3397us (97733us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762910.595543:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 896 00000100:00100000:0.0:1789762910.595552:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383839042334 time 383839177214 state 0x2 ref 0 00000100:00100000:2.0:1789762910.597531:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762910.597569:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067509248 00000100:00100000:0.0:1789762910.597577:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067509248 00000100:00100000:0.0:1789762910.597611:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383839042334 time 383839177214 state 0x0 ref 1 00000100:00100000:0.0:1789762910.597622:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383939177214 time 383839177214 state 0x1 ref 1 00000100:00100000:0.0:1789762910.692180:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383939177214 time 383939246827 state 0x0 ref 1 00000100:00100000:0.0:1789762910.692208:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8ede680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067509248:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762910.692246:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177119 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.692260:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762910.692264:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762910.692268:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177119 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.692290:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177119 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.695310:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177119 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.695325:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177119 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.695337:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177119 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.695345:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762910.695349:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177119 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.695357:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177119 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.695383:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177119 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.695388:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177119 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762910.695436:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8ede680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067509248:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3239us (97905us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762910.695443:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 897 00000100:00100000:0.0:1789762910.695452:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383939177214 time 383939246827 state 0x2 ref 0 00000100:00100000:2.0:1789762910.697569:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762910.697617:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067509376 00000100:00100000:1.0:1789762910.697632:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067509376 00000100:00100000:1.0:1789762910.697668:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 383939177214 time 383939246827 state 0x0 ref 1 00000100:00100000:1.0:1789762910.697680:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384039246827 time 383939246827 state 0x1 ref 1 00000100:00100000:2.0:1789762910.720083:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c8edce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074727680:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762910.720117:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d192c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074728192:0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.720122:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8a9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074728320:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762910.720153:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762910.720173:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762910.720179:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762910.720209:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff881f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074728064:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762910.720219:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f58700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074728832:0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.720222:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074728704:0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.720222:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f58380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074728448:0@lo:400:kworker.0 00000100:00100000:2.0:1789762910.720237:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762910.720259:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff8abb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074727808:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762910.720320:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762910.720327:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762910.720332:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074727680 00000100:00100000:2.0:1789762910.720334:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762910.720338:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074727680 00000100:00100000:0.0:1789762910.720379:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074729216:0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.720383:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074729344:0@lo:400:kworker.0 00000100:00100000:2.0:1789762910.720388:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff698a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074727936:0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.720399:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074728320 00000100:00100000:3.0:1789762910.720404:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074728320 00000100:00100000:0.0:1789762910.720419:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762910.720426:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762910.720432:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074728064 00000100:00100000:3.0:1789762910.720437:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074728064 00000100:00100000:2.0:1789762910.720445:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762910.720447:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074729728:0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.720460:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074727808 00000100:00100000:3.0:1789762910.720464:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074727808 00000100:00100000:1.0:1789762910.720476:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074728192 00000100:00100000:1.0:1789762910.720481:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074728192 00000100:00100000:0.0:1789762910.720487:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762910.720496:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074728576:0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.720497:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 234 00000100:00100000:3.0:1789762910.720513:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8eddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074727680:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762910.720517:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074728832 00000100:00100000:1.0:1789762910.720520:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074728832 00000100:00100000:2.0:1789762910.720531:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:3.0:1789762910.720548:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762910.720551:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074729216 00000100:00100000:2.0:1789762910.720552:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074729088:0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.720556:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074729216 00000100:00100000:1.0:1789762910.720558:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074728576 00000100:00100000:1.0:1789762910.720563:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074728576 00000100:00100000:1.0:1789762910.720599:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 572 00000100:00100000:0.0:1789762910.720610:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 573 00000100:00100000:1.0:1789762910.720613:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074728192:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.720614:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8eddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074727680:12345-0@lo:13:osp-pre-0-1.0 Request processed in 99us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762910.720615:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762910.720626:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074728832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.720626:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 234 00000100:00100000:2.0:1789762910.720641:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074729600:0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.720652:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:3.0:1789762910.720661:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074728320:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762910.720673:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:1.0:1789762910.720682:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074728192:12345-0@lo:400:kworker.0 Request processed in 75us (510us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762910.720690:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762910.720692:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074728832:12345-0@lo:400:kworker.0 Request processed in 67us (369us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762910.720699:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 572 00000100:00100000:3.0:1789762910.720705:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074728320:12345-0@lo:13:osp-pre-0-0.0 Request processed in 44us (527us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762910.720707:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 573 00000100:00100000:3.0:1789762910.720714:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00000100:00100000:1.0:1789762910.720725:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074729088 00000100:00100000:1.0:1789762910.720730:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074729088 00000100:00100000:0.0:1789762910.720732:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 574 00000100:00100000:3.0:1789762910.720733:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 236 00000100:00100000:3.0:1789762910.720746:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074728064:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762910.720747:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c8edce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074727680:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762910.720755:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff880a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074729216:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.720763:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 575 00000100:00100000:1.0:1789762910.720774:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074728576:12345-0@lo:400:kworker.0 00002000:00000024:3.0:1789762910.720784:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762910.720794:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1789762910.720803:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff880a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074729216:12345-0@lo:400:kworker.0 Request processed in 60us (382us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762910.720808:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074727808:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762910.720819:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 574 00002000:00000024:2.0:1789762910.720824:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762910.720832:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074728576:12345-0@lo:400:kworker.0 Request processed in 61us (301us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762910.720845:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074728064:12345-0@lo:13:osp-pre-1-1.0 Request processed in 100us (604us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762910.720848:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 576 00000100:00100000:1.0:1789762910.720855:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 575 00000100:00100000:3.0:1789762910.720856:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 236 00000100:00100000:0.0:1789762910.720860:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074729088:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762910.720876:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074727808:12345-0@lo:13:osp-pre-1-0.0 Request processed in 74us (543us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762910.720882:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00000100:00100000:1.0:1789762910.720900:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074728704 00000100:00100000:1.0:1789762910.720904:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074728704 00000100:00080000:3.0:1789762910.720905:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762910.720908:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074729088:12345-0@lo:400:kworker.0 Request processed in 51us (299us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762910.720911:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762910.720915:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074727936 00000100:00080000:3.0:1789762910.720939:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762910.720940:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074727936 00000100:00100000:0.0:1789762910.720941:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 576 00000100:00080000:3.0:1789762910.720944:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762910.720955:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:1.0:1789762910.720967:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074728704:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.720967:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762910.720990:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:2.0:1789762910.721000:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb93a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074727936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.721004:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074729344 00000100:00100000:0.0:1789762910.721011:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074729344 00000100:00100000:3.0:1789762910.721012:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074728960:0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.721016:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074728704:12345-0@lo:400:kworker.0 Request processed in 53us (692us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762910.721038:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00100000:1.0:1789762910.721057:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074728448 00000100:00100000:2.0:1789762910.721059:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb93a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074727936:12345-0@lo:400:kworker.0 Request processed in 60us (615us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762910.721061:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074728448 00000100:00100000:0.0:1789762910.721063:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074729728 00000100:00100000:3.0:1789762910.721068:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762910.721069:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074729728 00000100:00100000:2.0:1789762910.721072:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00000100:00100000:1.0:1789762910.721094:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:3.0:1789762910.721099:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074729472:0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.721104:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074728448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.721114:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074729600 00000100:00100000:0.0:1789762910.721120:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074729600 00000100:00100000:2.0:1789762910.721122:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:2.0:1789762910.721132:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074729344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.721149:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 236 00000100:00100000:3.0:1789762910.721157:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762910.721162:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074728448:12345-0@lo:400:kworker.0 Request processed in 57us (197us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762910.721163:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff882680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074729728:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.721173:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00000100:00100000:2.0:1789762910.721181:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074729344:12345-0@lo:400:kworker.0 Request processed in 50us (756us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762910.721188:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00000100:00100000:1.0:1789762910.721196:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074728960 00000100:00100000:3.0:1789762910.721199:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f58380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074728448:0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.721201:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074728960 00000100:00100000:2.0:1789762910.721207:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074729472 00000100:00100000:2.0:1789762910.721212:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074729472 00000100:00100000:0.0:1789762910.721218:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff882680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074729728:12345-0@lo:400:kworker.0 Request processed in 59us (734us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762910.721228:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 236 00000100:00100000:2.0:1789762910.721242:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:1.0:1789762910.721252:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 577 00000100:00100000:0.0:1789762910.721257:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:2.0:1789762910.721260:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074729600:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789762910.721265:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762910.721266:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074728960:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.721273:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074729472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762910.721296:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074729600:12345-0@lo:400:kworker.0 Request processed in 39us (612us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762910.721302:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00000100:00100000:3.0:1789762910.721309:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074728576:0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.721326:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074729088:0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.721333:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074728960:12345-0@lo:400:kworker.0 Request processed in 70us (265us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762910.721335:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074729472:12345-0@lo:400:kworker.0 Request processed in 68us (178us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762910.721344:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 577 00000100:00100000:3.0:1789762910.721347:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074729600:0@lo:400:kworker.0 00000100:00100000:2.0:1789762910.721354:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074728960:0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.721365:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00100000:3.0:1789762910.721379:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff881f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074728064:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762910.721380:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074729472:0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.721391:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f58700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074728832:0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.721411:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074729344:0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.721415:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8abb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074727808:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762910.721433:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff8a9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074728320:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762910.721440:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074728704:0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.721440:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff698a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074727936:0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.721457:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074729216:0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.721470:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074729728:0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.721479:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d192c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074728192:0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.792243:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384039246827 time 384039308152 state 0x0 ref 1 00000100:00100000:3.0:1789762910.792279:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067509376:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762910.792317:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177120 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.792332:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762910.792337:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762910.792343:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177120 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.792361:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177120 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.795876:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177120 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.795887:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177120 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.795894:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177120 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.795900:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762910.795905:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177120 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.795911:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177120 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.795942:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177120 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.795947:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177120 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762910.796015:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067509376:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3755us (98445us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762910.796039:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 898 00000100:00100000:3.0:1789762910.796060:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384039246827 time 384039308152 state 0x2 ref 0 00000100:00100000:1.0:1789762910.798366:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762910.798422:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067509504 00000100:00100000:0.0:1789762910.798430:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067509504 00000100:00100000:0.0:1789762910.798467:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384039246827 time 384039308152 state 0x0 ref 1 00000100:00100000:0.0:1789762910.798478:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384139308152 time 384039308152 state 0x1 ref 1 00010000:00010000:1.0:1789762910.848189:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762910.892352:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384139308152 time 384139415434 state 0x0 ref 1 00000100:00100000:3.0:1789762910.892382:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8ccf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067509504:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762910.892421:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177127 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.892436:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762910.892440:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762910.892445:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177127 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.892457:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177127 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.895509:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177127 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.895525:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177127 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.895537:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177127 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.895547:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762910.895553:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177127 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.895564:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177127 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.895581:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177127 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.895588:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177127 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762910.895648:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8ccf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067509504:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3278us (97280us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762910.895660:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 899 00000100:00100000:3.0:1789762910.895672:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384139308152 time 384139415434 state 0x2 ref 0 00000100:00100000:0.0:1789762910.898095:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762910.898154:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067509632 00000100:00100000:1.0:1789762910.898161:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067509632 00000100:00100000:1.0:1789762910.898203:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384139308152 time 384139415434 state 0x0 ref 1 00000100:00100000:1.0:1789762910.898214:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384239415434 time 384139415434 state 0x1 ref 1 00000100:00100000:0.0:1789762910.992429:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384239415434 time 384239489694 state 0x0 ref 1 00000100:00100000:0.0:1789762910.992460:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff881f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067509632:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762910.992507:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17712e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.992521:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762910.992525:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762910.992530:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17712e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.992545:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17712e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.995402:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17712e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.995413:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17712e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.995424:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17712e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.995433:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762910.995439:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17712e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.995451:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17712e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.995469:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17712e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.995475:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17712e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762910.995531:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff881f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067509632:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3084us (97437us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762910.995539:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 900 00000100:00100000:0.0:1789762910.995548:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384239415434 time 384239489694 state 0x2 ref 0 00000100:00100000:0.0:1789762910.997629:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762910.997669:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067509760 00000100:00100000:3.0:1789762910.997676:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067509760 00000100:00100000:3.0:1789762910.997712:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384239415434 time 384239489694 state 0x0 ref 1 00000100:00100000:3.0:1789762910.997722:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384339489694 time 384239489694 state 0x1 ref 1 00000100:00100000:1.0:1789762911.092523:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384339489694 time 384339579292 state 0x0 ref 1 00000100:00100000:1.0:1789762911.092561:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d192c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067509760:12345-192.168.203.37@tcp:3:dd.500 00000100:00080000:3.0:1789762911.092583:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762911.092596:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000840161f6/0xdf5a7ac59f177135 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.092611:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762911.092614:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762911.092624:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000840161f6/0xdf5a7ac59f177135 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.092636:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000840161f6/0xdf5a7ac59f177135 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.095412:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000840161f6/0xdf5a7ac59f177135 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.095422:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000840161f6/0xdf5a7ac59f177135 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.095429:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000840161f6/0xdf5a7ac59f177135 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.095435:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a6a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762911.095438:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000840161f6/0xdf5a7ac59f177135 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.095444:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000840161f6/0xdf5a7ac59f177135 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.095455:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000840161f6/0xdf5a7ac59f177135 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.095461:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000840161f6/0xdf5a7ac59f177135 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762911.095504:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d192c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067509760:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2965us (97875us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762911.095511:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 901 00000100:00100000:1.0:1789762911.095518:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384339489694 time 384339579292 state 0x2 ref 0 00000100:00100000:1.0:1789762911.097495:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762911.097545:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067509888 00000100:00100000:3.0:1789762911.097555:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067509888 00000100:00100000:3.0:1789762911.097594:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384339489694 time 384339579292 state 0x0 ref 1 00000100:00100000:3.0:1789762911.097605:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384439579292 time 384339579292 state 0x1 ref 1 00000100:00100000:0.0:1789762911.192621:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384439579292 time 384439686179 state 0x0 ref 1 00000100:00100000:0.0:1789762911.192653:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067509888:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762911.192698:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17713c lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.192712:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762911.192716:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762911.192720:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17713c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.192732:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17713c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.195720:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17713c lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.195733:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17713c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.195742:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17713c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.195748:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762911.195751:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17713c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.195758:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17713c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.195771:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17713c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.195776:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17713c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762911.195823:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067509888:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3186us (98328us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762911.195831:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 902 00000100:00100000:0.0:1789762911.195839:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384439579292 time 384439686179 state 0x2 ref 0 00000100:00100000:2.0:1789762911.197982:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762911.198014:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067510016 00000100:00100000:0.0:1789762911.198034:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067510016 00000100:00100000:0.0:1789762911.198070:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384439579292 time 384439686179 state 0x0 ref 1 00000100:00100000:0.0:1789762911.198084:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384539686179 time 384439686179 state 0x1 ref 1 00000100:00100000:0.0:1789762911.292703:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384539686179 time 384539769006 state 0x0 ref 1 00000100:00100000:0.0:1789762911.292732:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067510016:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762911.292770:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177143 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.292792:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762911.292795:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762911.292800:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177143 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.292812:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177143 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.295737:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177143 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.295746:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177143 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.295752:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177143 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.295762:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762911.295766:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177143 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.295773:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177143 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.295786:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177143 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.295791:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f177143 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762911.295836:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067510016:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3118us (97855us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762911.295843:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 903 00000100:00100000:0.0:1789762911.295851:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384539686179 time 384539769006 state 0x2 ref 0 00000100:00100000:1.0:1789762911.297942:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762911.297982:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067510144 00000100:00100000:0.0:1789762911.297991:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067510144 00000100:00100000:0.0:1789762911.298040:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384539686179 time 384539769006 state 0x0 ref 1 00000100:00100000:0.0:1789762911.298051:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384639769006 time 384539769006 state 0x1 ref 1 00000100:00100000:1.0:1789762911.392851:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384639769006 time 384639914454 state 0x0 ref 1 00000100:00100000:1.0:1789762911.392881:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff8a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067510144:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762911.392955:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17714a lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.392970:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762911.392973:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762911.392978:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17714a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.392990:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17714a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.396373:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17714a lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.396387:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17714a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.396395:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17714a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.396401:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762911.396413:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17714a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.396421:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17714a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.396436:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17714a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.396442:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17714a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762911.396492:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff8a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067510144:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3624us (98551us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762911.396499:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 904 00000100:00100000:1.0:1789762911.396507:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384639769006 time 384639914454 state 0x2 ref 0 00000100:00100000:1.0:1789762911.398649:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762911.398723:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067510272 00000100:00100000:0.0:1789762911.398733:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067510272 00000100:00100000:0.0:1789762911.398775:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384639769006 time 384639914454 state 0x0 ref 1 00000100:00100000:0.0:1789762911.398786:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384739914454 time 384639914454 state 0x1 ref 1 00000100:00100000:3.0:1789762911.493007:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384739914454 time 384740065878 state 0x0 ref 1 00000100:00100000:3.0:1789762911.493058:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067510272:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762911.493103:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177151 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.493121:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762911.493127:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762911.493134:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177151 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.493163:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177151 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.496209:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177151 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.496223:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177151 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.496234:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177151 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.496249:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762911.496255:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177151 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.496266:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177151 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.496284:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177151 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.496293:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177151 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762911.496348:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067510272:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3305us (97698us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762911.496360:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 905 00000100:00100000:3.0:1789762911.496371:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384739914454 time 384740065878 state 0x2 ref 0 00000100:00100000:1.0:1789762911.498393:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762911.498437:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067510400 00000100:00100000:0.0:1789762911.498446:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067510400 00000100:00100000:0.0:1789762911.498482:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384739914454 time 384740065878 state 0x0 ref 1 00000100:00100000:0.0:1789762911.498493:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384840065878 time 384740065878 state 0x1 ref 1 00000100:00100000:3.0:1789762911.593163:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384840065878 time 384840227898 state 0x0 ref 1 00000100:00100000:3.0:1789762911.593191:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067510400:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762911.593230:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177158 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.593250:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762911.593253:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762911.593258:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177158 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.593269:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177158 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.596573:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177158 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.596584:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177158 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.596592:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177158 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.596598:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762911.596601:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177158 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.596608:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177158 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.596622:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177158 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.596631:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177158 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762911.596678:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067510400:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3501us (98287us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762911.596685:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 906 00000100:00100000:3.0:1789762911.596693:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384840065878 time 384840227898 state 0x2 ref 0 00000100:00100000:1.0:1789762911.598676:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762911.598721:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067510528 00000100:00100000:0.0:1789762911.598729:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067510528 00000100:00100000:0.0:1789762911.598763:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384840065878 time 384840227898 state 0x0 ref 1 00000100:00100000:0.0:1789762911.598773:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384940227898 time 384840227898 state 0x1 ref 1 00000100:00100000:2.0:1789762911.693331:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384940227898 time 384940385706 state 0x0 ref 1 00000100:00100000:2.0:1789762911.693371:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067510528:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762911.693425:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fd1158b6/0xdf5a7ac59f17715f lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.693447:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762911.693452:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762911.693461:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd1158b6/0xdf5a7ac59f17715f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.693480:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd1158b6/0xdf5a7ac59f17715f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.697217:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fd1158b6/0xdf5a7ac59f17715f lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.697240:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd1158b6/0xdf5a7ac59f17715f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.697269:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd1158b6/0xdf5a7ac59f17715f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.697281:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d98e6200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762911.697288:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd1158b6/0xdf5a7ac59f17715f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.697303:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd1158b6/0xdf5a7ac59f17715f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.697348:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd1158b6/0xdf5a7ac59f17715f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.697362:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd1158b6/0xdf5a7ac59f17715f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762911.697434:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067510528:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4080us (98758us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762911.697468:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 907 00000100:00100000:2.0:1789762911.697479:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384940227898 time 384940385706 state 0x2 ref 0 00000100:00100000:2.0:1789762911.700242:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762911.700306:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067510656 00000100:00100000:3.0:1789762911.700316:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067510656 00000100:00100000:3.0:1789762911.700383:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 384940227898 time 384940385706 state 0x0 ref 1 00000100:00100000:3.0:1789762911.700399:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385040385706 time 384940385706 state 0x1 ref 1 00000100:00100000:1.0:1789762911.793448:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385040385706 time 385040513463 state 0x0 ref 1 00000100:00100000:1.0:1789762911.793479:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067510656:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762911.793524:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f177166 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.793541:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762911.793546:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762911.793553:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f177166 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.793570:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f177166 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.796947:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f177166 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.796957:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f177166 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.796963:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f177166 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.796976:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762911.796980:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f177166 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.796990:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f177166 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.797020:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f177166 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.797030:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f177166 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762911.797079:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067510656:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3615us (96840us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762911.797097:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 908 00000100:00100000:1.0:1789762911.797105:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385040385706 time 385040513463 state 0x2 ref 0 00000100:00100000:0.0:1789762911.799243:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762911.799282:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067510784 00000100:00100000:3.0:1789762911.799291:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067510784 00000100:00100000:3.0:1789762911.799333:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385040385706 time 385040513463 state 0x0 ref 1 00000100:00100000:3.0:1789762911.799348:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385140513463 time 385040513463 state 0x1 ref 1 00010000:00010000:2.0:1789762911.872052:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762911.893576:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385140513463 time 385140637781 state 0x0 ref 1 00000100:00100000:1.0:1789762911.893612:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067510784:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762911.893654:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17716d lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.893671:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762911.893677:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762911.893684:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17716d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.893698:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17716d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.896774:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17716d lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.896789:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17716d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.896807:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17716d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.896816:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c9af7200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762911.896821:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17716d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.896831:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17716d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.896849:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17716d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.896859:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001e6cb2f3/0xdf5a7ac59f17716d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762911.896958:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067510784:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3324us (97675us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762911.896970:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 909 00000100:00100000:1.0:1789762911.896982:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385140513463 time 385140637781 state 0x2 ref 0 00000100:00100000:2.0:1789762911.899198:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762911.899246:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067510912 00000100:00100000:0.0:1789762911.899255:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067510912 00000100:00100000:0.0:1789762911.899293:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385140513463 time 385140637781 state 0x0 ref 1 00000100:00100000:0.0:1789762911.899304:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385240637781 time 385140637781 state 0x1 ref 1 00000100:00100000:0.0:1789762911.993674:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385240637781 time 385240733501 state 0x0 ref 1 00000100:00100000:0.0:1789762911.993711:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067510912:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762911.993748:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f177174 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.993763:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762911.993766:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762911.993771:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f177174 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.993782:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f177174 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.997073:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f177174 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.997083:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f177174 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.997091:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f177174 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.997096:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762911.997100:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f177174 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.997107:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f177174 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.997123:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f177174 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.997129:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f177174 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762911.997177:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067510912:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3481us (97980us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762911.997184:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 910 00000100:00100000:0.0:1789762911.997191:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385240637781 time 385240733501 state 0x2 ref 0 00000100:00100000:2.0:1789762911.999680:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762911.999748:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067511040 00000100:00100000:1.0:1789762911.999758:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067511040 00000100:00100000:1.0:1789762911.999800:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385240637781 time 385240733501 state 0x0 ref 1 00000100:00100000:1.0:1789762911.999818:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385340733501 time 385240733501 state 0x1 ref 1 00000100:00100000:0.0:1789762912.093816:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385340733501 time 385340875666 state 0x0 ref 1 00000100:00100000:0.0:1789762912.093853:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067511040:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762912.093898:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17717b lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.093954:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762912.093960:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762912.093968:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17717b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.093983:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17717b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.097155:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17717b lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.097168:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17717b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.097177:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17717b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.097185:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d1769600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762912.097189:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17717b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.097198:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17717b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.097216:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17717b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.097225:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bb4a15d/0xdf5a7ac59f17717b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762912.097280:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067511040:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3446us (97602us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762912.097287:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 911 00000100:00100000:0.0:1789762912.097295:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385340733501 time 385340875666 state 0x2 ref 0 00000100:00100000:1.0:1789762912.099299:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762912.099377:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067511168 00000100:00100000:0.0:1789762912.099385:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067511168 00000100:00100000:0.0:1789762912.099421:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385340733501 time 385340875666 state 0x0 ref 1 00000100:00100000:0.0:1789762912.099432:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385440875666 time 385340875666 state 0x1 ref 1 00000100:00100000:3.0:1789762912.193914:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385440875666 time 385440965326 state 0x0 ref 1 00000100:00100000:3.0:1789762912.193989:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067511168:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762912.194069:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177182 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.194097:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762912.194103:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762912.194111:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177182 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.194131:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177182 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.198467:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177182 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.198486:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177182 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.198494:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177182 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.198501:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762912.198504:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177182 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.198511:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177182 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.198533:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177182 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.198539:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177182 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762912.198601:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067511168:12345-192.168.203.37@tcp:3:dd.500 Request processed in 4623us (99302us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762912.198610:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 912 00000100:00100000:3.0:1789762912.198620:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385440875666 time 385440965326 state 0x2 ref 0 00000100:00100000:1.0:1789762912.200763:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762912.200801:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067511296 00000100:00100000:0.0:1789762912.200809:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067511296 00000100:00100000:0.0:1789762912.200850:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385440875666 time 385440965326 state 0x0 ref 1 00000100:00100000:0.0:1789762912.200861:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385540965326 time 385440965326 state 0x1 ref 1 00000100:00100000:3.0:1789762912.294021:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385540965326 time 385541072203 state 0x0 ref 1 00000100:00100000:3.0:1789762912.294066:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067511296:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762912.294109:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177189 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.294123:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762912.294126:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762912.294131:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177189 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.294141:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177189 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.297277:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177189 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.297286:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177189 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.297293:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177189 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.297298:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762912.297302:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177189 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.297307:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177189 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.297319:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177189 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.297324:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f177189 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762912.297371:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067511296:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3332us (96610us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762912.297387:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 913 00000100:00100000:3.0:1789762912.297401:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385540965326 time 385541072203 state 0x2 ref 0 00000100:00100000:1.0:1789762912.299503:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762912.299587:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067511424 00000100:00100000:0.0:1789762912.299596:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067511424 00000100:00100000:0.0:1789762912.299656:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385540965326 time 385541072203 state 0x0 ref 1 00000100:00100000:0.0:1789762912.299666:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385641072203 time 385541072203 state 0x1 ref 1 00000100:00100000:2.0:1789762912.394145:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385641072203 time 385641209476 state 0x0 ref 1 00000100:00100000:2.0:1789762912.394173:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff6ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067511424:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762912.394209:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f177190 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.394223:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762912.394226:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762912.394231:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f177190 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.394242:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f177190 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.397307:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f177190 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.397317:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f177190 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.397325:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f177190 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.397331:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5932a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762912.397334:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f177190 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.397359:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f177190 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.397373:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f177190 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.397378:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000170794d2/0xdf5a7ac59f177190 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762912.397431:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff6ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067511424:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3270us (97930us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762912.397438:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 914 00000100:00100000:2.0:1789762912.397445:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385641072203 time 385641209476 state 0x2 ref 0 00000100:00100000:1.0:1789762912.399657:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762912.399710:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067511552 00000100:00100000:3.0:1789762912.399723:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067511552 00000100:00100000:3.0:1789762912.399783:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385641072203 time 385641209476 state 0x0 ref 1 00000100:00100000:3.0:1789762912.399798:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385741209476 time 385641209476 state 0x1 ref 1 00000100:00100000:1.0:1789762912.494241:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385741209476 time 385741305983 state 0x0 ref 1 00000100:00100000:1.0:1789762912.494269:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067511552:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762912.494309:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f177197 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762912.494323:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762912.494327:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762912.494331:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f177197 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762912.494342:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f177197 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.496996:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f177197 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.497020:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f177197 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.497028:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f177197 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.497034:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795a600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762912.497038:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f177197 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.497044:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f177197 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.497058:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f177197 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.497063:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000366fe40/0xdf5a7ac59f177197 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762912.497112:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067511552:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2856us (97457us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762912.497119:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 915 00000100:00100000:0.0:1789762912.497127:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385741209476 time 385741305983 state 0x2 ref 0 00000100:00100000:0.0:1789762912.499160:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762912.499201:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067511680 00000100:00100000:3.0:1789762912.499208:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067511680 00000100:00100000:3.0:1789762912.499244:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385741209476 time 385741305983 state 0x0 ref 1 00000100:00100000:3.0:1789762912.499255:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385841305983 time 385741305983 state 0x1 ref 1 00000100:00100000:1.0:1789762912.594344:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385841305983 time 385841410208 state 0x0 ref 1 00000100:00100000:1.0:1789762912.594372:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eaf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067511680:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762912.594412:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17719e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762912.594431:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762912.594436:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762912.594443:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17719e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762912.594458:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17719e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.597652:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17719e lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.597663:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17719e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.597671:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17719e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.597677:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37f795b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762912.597680:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17719e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.597696:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17719e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.597710:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17719e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.597716:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e44630a/0xdf5a7ac59f17719e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762912.597766:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eaf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067511680:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3407us (98607us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762912.597773:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 916 00000100:00100000:2.0:1789762912.597781:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385841305983 time 385841410208 state 0x2 ref 0 00000100:00100000:1.0:1789762912.600052:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762912.600083:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067511808 00000100:00100000:2.0:1789762912.600090:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067511808 00000100:00100000:2.0:1789762912.600124:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385841305983 time 385841410208 state 0x0 ref 1 00000100:00100000:2.0:1789762912.600134:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385941410208 time 385841410208 state 0x1 ref 1 00000100:00100000:3.0:1789762912.694467:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385941410208 time 385941532063 state 0x0 ref 1 00000100:00100000:3.0:1789762912.694495:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067511808:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762912.694532:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771a5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.694545:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762912.694549:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762912.694554:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771a5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.694565:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771a5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.697587:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771a5 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.697597:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771a5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.697604:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771a5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.697608:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762912.697612:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771a5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.697618:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771a5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.697631:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771a5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.697636:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771a5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762912.697684:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067511808:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3201us (97632us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762912.697691:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 917 00000100:00100000:3.0:1789762912.697699:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385941410208 time 385941532063 state 0x2 ref 0 00000100:00100000:0.0:1789762912.700091:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762912.700137:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067511936 00000100:00100000:1.0:1789762912.700146:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067511936 00000100:00100000:1.0:1789762912.700184:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 385941410208 time 385941532063 state 0x0 ref 1 00000100:00100000:1.0:1789762912.700194:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386041532063 time 385941532063 state 0x1 ref 1 00000100:00100000:3.0:1789762912.768100:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789762912.768122:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074729856:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762912.768222:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762912.768278:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074729856 00000100:00100000:0.0:1789762912.768289:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074729856 00000100:00100000:0.0:1789762912.768350:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 388 00000100:00100000:0.0:1789762912.768364:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074729856:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762912.768424:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762912.768485:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074729856:12345-0@lo:41:osp-pre-1-0.0 Request processed in 119us (257us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762912.768492:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 388 00000100:00100000:3.0:1789762912.768531:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074729856:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762912.794581:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386041532063 time 386041647288 state 0x0 ref 1 00000100:00100000:0.0:1789762912.794608:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eadf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067511936:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762912.794645:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771ac lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.794659:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762912.794663:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762912.794668:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771ac lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.794680:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771ac lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.797538:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771ac lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.797550:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771ac lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.797557:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771ac lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.797563:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d920ca00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762912.797568:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771ac lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.797575:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771ac lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.797590:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771ac lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.797595:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771ac lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762912.797641:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eadf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067511936:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3046us (97550us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762912.797650:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 918 00000100:00100000:0.0:1789762912.797658:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386041532063 time 386041647288 state 0x2 ref 0 00000100:00100000:0.0:1789762912.799983:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762912.800039:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067512064 00000100:00100000:2.0:1789762912.800054:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067512064 00000100:00100000:2.0:1789762912.800098:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386041532063 time 386041647288 state 0x0 ref 1 00000100:00100000:2.0:1789762912.800111:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386141647288 time 386041647288 state 0x1 ref 1 00000100:00100000:0.0:1789762912.894687:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386141647288 time 386141751895 state 0x0 ref 1 00000100:00100000:0.0:1789762912.894714:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067512064:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762912.894752:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771b3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.894766:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762912.894769:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762912.894774:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771b3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762912.894785:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771b3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.896085:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762912.897732:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771b3 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762912.897747:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771b3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762912.897757:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771b3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762912.897766:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d920ca00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762912.897773:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771b3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762912.897784:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771b3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762912.897804:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771b3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762912.897814:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ab78721/0xdf5a7ac59f1771b3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762912.897873:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067512064:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3169us (97889us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762912.897884:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 919 00000100:00100000:1.0:1789762912.897896:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386141647288 time 386141751895 state 0x2 ref 0 00000100:00100000:1.0:1789762912.899976:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762912.900017:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067512192 00000100:00100000:3.0:1789762912.900027:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067512192 00000100:00100000:3.0:1789762912.900071:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386141647288 time 386141751895 state 0x0 ref 1 00000100:00100000:3.0:1789762912.900082:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386241751895 time 386141751895 state 0x1 ref 1 00000100:00100000:3.0:1789762912.994745:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386241751895 time 386241811260 state 0x0 ref 1 00000100:00100000:3.0:1789762912.994777:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c9df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067512192:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:3.0:1789762912.994819:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771ba lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.994833:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762912.994836:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762912.994841:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771ba lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.994852:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771ba lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.997817:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771ba lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.997827:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771ba lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.997834:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771ba lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.997840:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762912.997844:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771ba lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.997851:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771ba lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.997865:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771ba lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.997870:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f1771ba lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762912.997952:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c9df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067512192:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3156us (97942us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762912.997963:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 920 00000100:00100000:3.0:1789762912.997972:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386241751895 time 386241811260 state 0x2 ref 0 00000100:00100000:0.0:1789762913.000567:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762913.000621:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067512320 00000100:00100000:1.0:1789762913.000631:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067512320 00000100:00100000:1.0:1789762913.000679:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386241751895 time 386241811260 state 0x0 ref 1 00000100:00100000:1.0:1789762913.000695:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386341811260 time 386241811260 state 0x1 ref 1 00000100:00100000:0.0:1789762913.094828:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386341811260 time 386341893461 state 0x0 ref 1 00000100:00100000:0.0:1789762913.094869:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067512320:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:0.0:1789762913.094907:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000f89aa4d/0xdf5a7ac59f1771c1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789762913.094907:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762913.094952:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762913.094957:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762913.094962:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f89aa4d/0xdf5a7ac59f1771c1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.094973:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f89aa4d/0xdf5a7ac59f1771c1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.097721:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000f89aa4d/0xdf5a7ac59f1771c1 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.097732:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f89aa4d/0xdf5a7ac59f1771c1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.097748:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f89aa4d/0xdf5a7ac59f1771c1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.097757:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c9bcc400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762913.097762:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f89aa4d/0xdf5a7ac59f1771c1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.097773:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f89aa4d/0xdf5a7ac59f1771c1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.097790:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f89aa4d/0xdf5a7ac59f1771c1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.097799:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f89aa4d/0xdf5a7ac59f1771c1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762913.097852:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067512320:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3009us (97285us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762913.097859:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 921 00000100:00100000:0.0:1789762913.097866:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386341811260 time 386341893461 state 0x2 ref 0 00000100:00100000:0.0:1789762913.099898:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762913.099976:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067512448 00000100:00100000:1.0:1789762913.099984:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067512448 00000100:00100000:1.0:1789762913.100033:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386341811260 time 386341893461 state 0x0 ref 1 00000100:00100000:1.0:1789762913.100045:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386441893461 time 386341893461 state 0x1 ref 1 00000100:00100000:1.0:1789762913.194901:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386441893461 time 386441966218 state 0x0 ref 1 00000100:00100000:1.0:1789762913.194953:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067512448:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:1.0:1789762913.194991:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1771c8 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.195018:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762913.195022:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762913.195028:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1771c8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.195040:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1771c8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.197756:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1771c8 lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.197768:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1771c8 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.197776:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1771c8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.197786:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762913.197790:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1771c8 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.197797:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1771c8 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.197810:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1771c8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.197815:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f1771c8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762913.197861:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067512448:12345-192.168.203.37@tcp:3:dd.500 Request processed in 2946us (97963us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762913.197868:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 922 00000100:00100000:1.0:1789762913.197876:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386441893461 time 386441966218 state 0x2 ref 0 00000100:00100000:1.0:1789762913.200268:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762913.200307:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067512576 00000100:00100000:3.0:1789762913.200315:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067512576 00000100:00100000:3.0:1789762913.200363:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386441893461 time 386441966218 state 0x0 ref 1 00000100:00100000:3.0:1789762913.200377:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386541966218 time 386441966218 state 0x1 ref 1 00000100:00100000:2.0:1789762913.295050:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386541966218 time 386542115721 state 0x0 ref 1 00000100:00100000:2.0:1789762913.295078:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067512576:12345-192.168.203.37@tcp:3:dd.500 00010000:00010000:2.0:1789762913.295117:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1771cf lrc: 3/1,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.295130:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762913.295134:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762913.295138:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1771cf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.295150:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1771cf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.298058:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1771cf lrc: 3/1,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.298067:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1771cf lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.298072:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1771cf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.298077:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caec00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762913.298080:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1771cf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.298100:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1771cf lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.298114:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1771cf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.298119:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1771cf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762913.298167:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067512576:12345-192.168.203.37@tcp:3:dd.500 Request processed in 3102us (97901us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762913.298174:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 923 00000100:00100000:2.0:1789762913.298181:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca132600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 386541966218 time 386542115721 state 0x2 ref 0 00000100:00100000:0.0:1789762913.323783:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762913.323855:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067512704 00000100:00100000:1.0:1789762913.323870:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067512704 00000100:00100000:1.0:1789762913.323975:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 68 00000100:00100000:1.0:1789762913.323986:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eafb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:6358:x1876702067512704:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762913.324015:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762913.324071:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b75faed6/0xdf5a7ac59f176f13 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca62 expref: 6 pid: 8053 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.324105:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762913.324183:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eafb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:6358:x1876702067512704:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 199us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762913.324191:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 68 00000100:00100000:0.0:1789762913.333092:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762913.333154:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067512832 00000100:00100000:2.0:1789762913.333163:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067512832 00000100:00080000:3.0:1789762913.333219:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762913.333231:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 332 00000100:00100000:2.0:1789762913.333254:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eaf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:13087:x1876702067512832:12345-192.168.203.37@tcp:101:bash.0 00010000:00010000:2.0:1789762913.333274:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762913.333336:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caec00/0xdf5a7ac59f1771d6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca69 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.333408:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1771dd lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.333445:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1771dd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.333455:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1771dd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762913.333488:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 16384 00000004:00200000:2.0:1789762913.333492:0:6420:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762913.333515:0:6420:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1771dd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.333542:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caec00/0xdf5a7ac59f1771d6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca69 expref: 11 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.333556:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1771dd lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca69 expref: 10 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.333563:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762913.333566:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762913.333568:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8cafc00, rc 0) 00000100:00100000:2.0:1789762913.333620:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eaf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:13087:x1876702067512832:12345-192.168.203.37@tcp:101:bash.0 Request processed in 375us (529us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762913.333626:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 332 00000100:00100000:2.0:1789762913.337480:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762913.337527:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067513088 00000100:00100000:3.0:1789762913.337534:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067513088 00000100:00100000:3.0:1789762913.337573:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 333 00000100:00100000:3.0:1789762913.337580:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:15228:x1876702067513088:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:3.0:1789762913.337595:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762913.337614:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f1771e4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca70 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762913.337693:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 16384 00010000:00010000:3.0:1789762913.338012:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f1771e4 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca70 expref: 12 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762913.338025:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762913.338027:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762913.338031:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f1771e4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca70 expref: 12 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762913.338040:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca315400, rc 0) 00000100:00100000:3.0:1789762913.338073:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15228:x1876702067513088:12345-192.168.203.37@tcp:101:rm.0 Request processed in 494us (595us total) trans 4294967438 rc 301/301 00000100:00100000:3.0:1789762913.338078:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 333 00000100:00100000:0.0:1789762913.339709:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762913.339783:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067513216 00000100:00100000:2.0:1789762913.339791:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067513216 00000100:00100000:2.0:1789762913.339825:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 97 00000100:00100000:2.0:1789762913.339832:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ead180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15228:x1876702067513216:12345-192.168.203.37@tcp:37:rm.0 00000100:00100000:2.0:1789762913.341903:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ead180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15228:x1876702067513216:12345-192.168.203.37@tcp:37:rm.0 Request processed in 2069us (2194us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762913.341911:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 97 00000100:00100000:1.0:1789762913.343186:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762913.343230:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067513472 00000100:00100000:0.0:1789762913.343236:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067513472 00000100:00100000:0.0:1789762913.343289:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 98 00000100:00100000:0.0:1789762913.343304:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff617800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15228:x1876702067513472:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:0.0:1789762913.343455:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff617800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:15228:x1876702067513472:12345-192.168.203.37@tcp:35:rm.0 Request processed in 152us (270us total) trans 4294967439 rc 0/0 00000100:00100000:0.0:1789762913.343462:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 98 00000100:00100000:0.0:1789762913.344372:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762913.344439:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067513728 00000100:00100000:2.0:1789762913.344446:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067513728 00000100:00100000:2.0:1789762913.344481:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 334 00000100:00100000:2.0:1789762913.344488:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eaca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:15228:x1876702067513728:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:2.0:1789762913.344500:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762913.344512:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f1771f2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca77 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.344562:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae800/0xdf5a7ac59f1771f9 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.344572:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae800/0xdf5a7ac59f1771f9 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.344579:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae800/0xdf5a7ac59f1771f9 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762913.344585:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 16384 00010000:00010000:2.0:1789762913.344670:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f1771f2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca77 expref: 15 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.344682:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae800/0xdf5a7ac59f1771f9 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca77 expref: 14 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.344687:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762913.344695:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1789762913.344697:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8cae800, rc 0) 00000100:00100000:2.0:1789762913.344733:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eaca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15228:x1876702067513728:12345-192.168.203.37@tcp:101:rm.0 Request processed in 245us (362us total) trans 4294967440 rc 0/0 00000100:00100000:2.0:1789762913.344738:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 334 00000100:00100000:2.0:1789762913.345643:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762913.345698:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067513856 00000100:00100000:0.0:1789762913.345705:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067513856 00000100:00100000:0.0:1789762913.345736:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 335 00000100:00100000:0.0:1789762913.345743:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15228:x1876702067513856:12345-192.168.203.37@tcp:41:rm.0 00000004:00000024:0.0:1789762913.345795:0:6418:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7614758912 avail 7603224576 00000100:00100000:0.0:1789762913.345844:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15228:x1876702067513856:12345-192.168.203.37@tcp:41:rm.0 Request processed in 101us (202us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762913.345851:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 335 00000100:00100000:1.0:1789762913.346678:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762913.346716:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067514112 00000100:00100000:2.0:1789762913.346723:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067514112 00000100:00100000:2.0:1789762913.346755:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 99 00000100:00100000:2.0:1789762913.346761:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15228:x1876702067514112:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:2.0:1789762913.346958:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:15228:x1876702067514112:12345-192.168.203.37@tcp:35:rm.0 Request processed in 197us (281us total) trans 4294967441 rc 0/0 00000100:00100000:2.0:1789762913.346965:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 99 00000100:00100000:1.0:1789762913.347760:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762913.347801:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067514368 00000100:00100000:2.0:1789762913.347805:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067514368 00000100:00100000:2.0:1789762913.347833:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 336 00000100:00100000:2.0:1789762913.347840:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:15228:x1876702067514368:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:2.0:1789762913.347864:0:13389:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789762913.347898:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36c9af7400/0xdf5a7ac59f176f05 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca5b expref: 15 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.347914:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315c00/0xdf5a7ac59f176c18 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca54 expref: 14 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.347942:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff9c37e37a7e00) 00010000:00010000:2.0:1789762913.347952:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1771dd lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca69 expref: 13 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.347968:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762913.347971:0:13389:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762913.347981:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177207 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.347987:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177207 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.347993:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177207 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.348028:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f17720e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.348034:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f17720e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.348039:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f17720e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.348281:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f177215 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.348288:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f177215 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.348293:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f177215 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789762913.349255:0:13389:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].8, 64 off8704 00000004:00080000:2.0:1789762913.349263:0:13389:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 386596344681ns, next at 387596343099ns, rc = 0 00010000:00010000:2.0:1789762913.349303:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf200/0xdf5a7ac59f17721c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349310:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf200/0xdf5a7ac59f17721c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349315:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf200/0xdf5a7ac59f17721c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349320:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf200/0xdf5a7ac59f17721c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349325:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf200/0xdf5a7ac59f17721c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349330:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf200/0xdf5a7ac59f17721c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349347:0:13389:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762913.349355:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f177215 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349359:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f177215 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349365:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f177215 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349370:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762913.349374:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f177215 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349381:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f177215 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.349392:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762913.349395:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762913.349399:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762913.349403:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf200/0xdf5a7ac59f17721c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349403:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f177215 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.349453:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762913.349454:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f177215 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.349463:0:6407:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf200/0xdf5a7ac59f17721c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.349473:0:6407:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf200/0xdf5a7ac59f17721c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349477:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177207 lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.349487:0:6407:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff9c37d3bee500) 00010000:00010000:2.0:1789762913.349489:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177207 lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.349492:0:6407:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762913.349497:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf200/0xdf5a7ac59f17721c lrc: 1/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.349518:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf200/0xdf5a7ac59f17721c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349519:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177207 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349545:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177207 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349553:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762913.349557:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177207 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.349559:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762913.349565:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762913.349569:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762913.349570:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177207 lrc: 5/0,0 mode: CW/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349583:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762913.349586:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762913.349589:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177207 lrc: 3/0,0 mode: --/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349602:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177207 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762913.349612:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d44f3cf1 reply state 00000000d3fc73dc transno 4294967442 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f17720e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349620:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f17720e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.349625:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x4009f790 (ffff9c37d3bee600) 00010000:00010000:2.0:1789762913.349627:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762913.349629:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762913.349673:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15228:x1876702067514368:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1832us (1913us total) trans 4294967442 rc 0/0 00000100:00100000:2.0:1789762913.349682:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 336 00000100:00100000:0.0:1789762913.350723:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762913.350798:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067514624 00000100:00100000:1.0:1789762913.350805:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067514624 00000100:00100000:1.0:1789762913.350838:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 100 00000100:00100000:1.0:1789762913.350845:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbe050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15228:x1876702067514624:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:1.0:1789762913.350985:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbe050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:15228:x1876702067514624:12345-192.168.203.37@tcp:35:rm.0 Request processed in 140us (266us total) trans 4294967443 rc 0/0 00000100:00100000:1.0:1789762913.350991:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 100 00000100:00100000:1.0:1789762913.351677:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762913.351720:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067514880 00000100:00100000:2.0:1789762913.351727:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067514880 00000100:00100000:2.0:1789762913.351761:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 337 00000100:00100000:2.0:1789762913.351767:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff617800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:15228:x1876702067514880:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:2.0:1789762913.351793:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789762913.351803:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae800/0xdf5a7ac59f1771f9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca77 expref: 15 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.351817:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315a00/0xdf5a7ac59f176bfc lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca4d expref: 14 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.351825:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762913.351842:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c5860c00/0xdf5a7ac59f176b9a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca38 expref: 13 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.351848:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762913.351850:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762913.351859:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177223 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.351866:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177223 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.351871:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177223 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.351878:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f17722a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.351883:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f17722a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.351888:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f17722a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352102:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177231 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352109:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177231 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352113:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177231 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762913.352413:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 16384 00010000:00010000:2.0:1789762913.352426:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177231 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352431:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177231 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352436:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177231 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352439:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafe00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762913.352441:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177231 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352445:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177231 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352451:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff9c37d3beef00) 00010000:00010000:2.0:1789762913.352453:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762913.352455:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177231 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352459:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177231 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352470:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177223 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352474:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177223 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352478:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177223 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352482:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177223 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352490:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762913.352492:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177223 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352496:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177223 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352501:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762913.352502:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762913.352504:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177223 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352508:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177223 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762913.352513:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e7f86b66 reply state 00000000f9ae6715 transno 4294967444 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f17722a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352520:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f17722a lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352524:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee200) 00010000:00010000:2.0:1789762913.352525:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762913.352527:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762913.352567:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff617800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15228:x1876702067514880:12345-192.168.203.37@tcp:36:rm.0 Request processed in 799us (891us total) trans 4294967444 rc 0/0 00000100:00100000:2.0:1789762913.352572:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 337 00000100:00100000:2.0:1789762913.792080:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074729984:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762913.792169:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762913.792222:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074729984 00000100:00100000:1.0:1789762913.792231:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074729984 00000100:00100000:1.0:1789762913.792292:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00100000:1.0:1789762913.792305:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074729984:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789762913.792348:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:1.0:1789762913.792433:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074729984:12345-0@lo:41:osp-pre-0-1.0 Request processed in 133us (270us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762913.792441:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00000100:00100000:2.0:1789762913.792460:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074729984:0@lo:41:osp-pre-0-1.0 00010000:00010000:1.0:1789762913.920119:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1789762914.265701:0:17344:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 387512781131ns, next at 387596343099ns, rc = 0 00000001:00080000:3.0:1789762914.275354:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967439 is committed 00000001:00080000:3.0:1789762914.275379:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967443 is committed 00000001:00080000:3.0:1789762914.275396:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967444 is committed 00000004:00080000:3.0:1789762914.275407:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 13 committed 00010000:00010000:0.0:1789762914.275420:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f17720e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.275429:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f17722a lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762914.275436:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 14 committed 00010000:00010000:0.0:1789762914.275444:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f17720e lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.275455:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f17722a lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762914.275460:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f17720e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762914.275482:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caee00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762914.275483:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f17722a lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762914.275490:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f17720e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.275492:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762914.275501:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f17722a lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762914.275504:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f17720e lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.275511:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f17722a lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762914.275526:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x4009f790 (ffff9c37d3bee600) 00010000:00010000:2.0:1789762914.275527:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3bee200) 00010000:00010000:0.0:1789762914.275537:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762914.275537:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762914.275548:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f17720e lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.275548:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f17722a lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762914.275558:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074730112:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1789762914.275575:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f17722a lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762914.275584:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f17720e lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762914.275633:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762914.275666:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074730112 00000100:00100000:2.0:1789762914.275676:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074730112 00000100:00080000:0.0:1789762914.275729:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762914.275743:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 201 00000100:00100000:2.0:1789762914.275755:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074730112:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1789762914.275782:0:9038:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:8 count 1 00002000:00010000:2.0:1789762914.275808:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x8:0x0] initial LVB size: 104857600, mtime: 0x6aad9d57, atime: 0x0, ctime: 0x6aad9d57, blocks: 0x3200b 00010000:00010000:2.0:1789762914.275822:0:9038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177238 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.275837:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=387s v=(0 0) 00010000:00010000:2.0:1789762914.275841:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762914.275847:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177238 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.275861:0:9038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177238 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.275868:0:9038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177238 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.275876:0:9038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177238 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.275902:0:9038:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762914.275955:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762914.275963:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177238 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762914.275973:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762914.275977:0:6408:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177238 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762914.275986:0:6408:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177238 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762914.276018:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177238 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762914.276026:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177238 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762914.276048:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762914.276052:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762914.276054:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762914.276491:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3424:x1876702074730112:12345-0@lo:6:osp-syn-0-0.0 Request processed in 736us (855us total) trans 4294968031 rc 0/0 00000100:00100000:2.0:1789762914.276501:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 201 00000004:00080000:3.0:1789762914.276535:0:3424:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37ff616680/2, rc 0, transno 4294968031 00000100:00100000:3.0:1789762914.276553:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074730112:0@lo:6:osp-syn-0-0.0 00000004:00080000:0.0:1789762914.276723:0:17344:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 387523803630ns, next at 369607186188ns, rc = 0 00000004:00080000:0.0:1789762914.276848:0:17344:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 387523929160ns, next at 0ns, rc = 0 00000004:00080000:0.0:1789762914.277090:0:17344:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 387524170887ns, next at 369607154287ns, rc = 0 00000004:00080000:3.0:1789762914.278577:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 7 committed 00000004:00080000:2.0:1789762914.279085:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 20 committed 00000004:00080000:3.0:1789762914.280423:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 20 committed 00000100:00100000:0.0:1789762914.760150:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762914.760203:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762914.760209:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067515008 00000100:00100000:2.0:1789762914.760224:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067515008 00000100:00100000:1.0:1789762914.760255:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067515264 00000100:00100000:1.0:1789762914.760265:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067515264 00000100:00100000:0.0:1789762914.760269:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762914.760301:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 377 00000100:00100000:2.0:1789762914.760314:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb938450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067515008:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762914.760356:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762914.760365:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067515136 00000100:00100000:1.0:1789762914.760372:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 578 00000100:00100000:3.0:1789762914.760373:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067515136 00000100:00100000:1.0:1789762914.760398:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eaf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067515264:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762914.760401:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb938450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067515008:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 89us (274us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762914.760421:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 377 00000100:00100000:3.0:1789762914.760429:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 579 00000100:00100000:3.0:1789762914.760440:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067515136:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762914.760468:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eaf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067515264:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 77us (261us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762914.760477:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 578 00000100:00100000:3.0:1789762914.760502:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067515136:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 59us (229us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762914.760515:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 579 00000100:00080000:1.0:1789762914.760538:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762914.760545:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067515392 00000100:00100000:2.0:1789762914.760551:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067515392 00000100:00100000:2.0:1789762914.760608:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 239 00000100:00100000:2.0:1789762914.760621:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eaea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067515392:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762914.760662:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eaea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067515392:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 43us (318us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762914.760668:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 239 00000100:00100000:0.0:1789762914.762960:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762914.763007:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762914.763011:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067515776 00000100:00100000:2.0:1789762914.763018:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067515776 00000100:00100000:0.0:1789762914.763058:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762914.763083:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067515648 00000100:00080000:3.0:1789762914.763086:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762914.763090:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067515648 00000100:00100000:0.0:1789762914.763110:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 580 00000100:00100000:0.0:1789762914.763129:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067515776:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:3.0:1789762914.763138:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762914.763152:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 581 00000100:00100000:2.0:1789762914.763171:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ead880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067515648:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762914.763183:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067515776:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 62us (221us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762914.763195:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 580 00000100:00100000:2.0:1789762914.763221:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ead880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067515648:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 56us (161us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762914.763229:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 581 00000100:00100000:1.0:1789762914.764241:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067515520 00000100:00100000:1.0:1789762914.764247:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067515520 00000100:00080000:3.0:1789762914.764296:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762914.764301:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 202 00000100:00100000:1.0:1789762914.764320:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ead500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2394:x1876702067515520:12345-192.168.203.37@tcp:9:kworker.0 00000020:00000080:1.0:1789762914.764340:0:8053:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xdf5a7ac59f17699b 00000100:00080000:1.0:1789762914.764350:0:8053:0:(import.c:65:import_set_state_nolock()) ffff9c37e1bef800 : changing import state from FULL to CLOSED 00010000:00010000:3.0:1789762914.764386:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00000004:1.0:1789762914.764390:0:8053:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967417 00000100:00100000:1.0:1789762914.764617:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ead500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+2:2394:x1876702067515520:12345-192.168.203.37@tcp:9:kworker.0 Request processed in 307us (1610us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762914.764627:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 202 00010000:00010000:0.0:1789762914.944098:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762915.841094:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074730368:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762915.841096:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eadc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074730496:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762915.841205:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762915.841207:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8c8c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074730240:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762915.841259:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762915.841303:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eaed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074730752:0@lo:400:kworker.0 00000100:00100000:3.0:1789762915.841310:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0ead180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074730880:0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.841317:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074730368 00000100:00100000:0.0:1789762915.841331:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074730368 00000100:00100000:2.0:1789762915.841346:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762915.841383:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb590a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074731264:0@lo:400:kworker.0 00000100:00100000:3.0:1789762915.841389:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762915.841427:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074730240 00000100:00100000:3.0:1789762915.841433:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb593100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074731392:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.841435:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762915.841438:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074730240 00000100:00100000:3.0:1789762915.841484:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762915.841492:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb590e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074731776:0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.841515:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:3.0:1789762915.841521:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb591500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074731904:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.841532:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762915.841539:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074730368:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762915.841575:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762915.841578:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00002000:00000024:0.0:1789762915.841595:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762915.841595:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074730624:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762915.841607:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1789762915.841617:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb593480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074731008:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.841646:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762915.841682:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb591180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074731136:0@lo:400:kworker.0 00000100:00100000:3.0:1789762915.841692:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762915.841705:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074730368:12345-0@lo:13:osp-pre-1-0.0 Request processed in 172us (497us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762915.841719:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00100000:3.0:1789762915.841727:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb590380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074731520:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.841738:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762915.841755:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074730624 00000100:00100000:0.0:1789762915.841762:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074730624 00000100:00100000:3.0:1789762915.841769:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762915.841776:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb592a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074731648:0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.841799:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 239 00000100:00100000:0.0:1789762915.841813:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074730240:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762915.841820:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb591880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074732032:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.841830:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:0.0:1789762915.841851:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:3.0:1789762915.841861:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762915.841871:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb591c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074732160:0@lo:400:kworker.0 00000100:00100000:3.0:1789762915.841958:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074730368:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762915.841969:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762915.841970:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074730240:12345-0@lo:13:osp-pre-0-1.0 Request processed in 153us (705us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762915.842007:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 239 00000100:00100000:3.0:1789762915.842043:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 240 00000100:00100000:2.0:1789762915.842049:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8c8c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074730240:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762915.842058:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074730624:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762915.842075:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762915.842110:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074731904 00000100:00100000:2.0:1789762915.842116:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074731904 00000100:00100000:1.0:1789762915.842131:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074730880 00000100:00100000:3.0:1789762915.842133:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074730624:12345-0@lo:13:osp-pre-0-0.0 Request processed in 79us (487us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762915.842143:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074730880 00000100:00100000:3.0:1789762915.842148:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 240 00000100:00100000:2.0:1789762915.842165:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074732032 00000100:00100000:2.0:1789762915.842171:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074732032 00000100:00080000:3.0:1789762915.842187:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789762915.842195:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762915.842198:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074731392 00000100:00100000:1.0:1789762915.842206:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074731392 00000100:00100000:0.0:1789762915.842207:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074731776 00000100:00100000:2.0:1789762915.842216:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074732160 00000100:00100000:0.0:1789762915.842217:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074731776 00000100:00100000:3.0:1789762915.842221:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074731136 00000100:00100000:2.0:1789762915.842224:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074732160 00000100:00100000:3.0:1789762915.842228:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074731136 00000100:00100000:2.0:1789762915.842256:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 240 00000100:00100000:1.0:1789762915.842271:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074731520 00000100:00100000:2.0:1789762915.842271:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074731904:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.842274:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074731648 00000100:00100000:1.0:1789762915.842282:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074731520 00000100:00100000:3.0:1789762915.842282:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 582 00000100:00100000:0.0:1789762915.842284:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074731648 00000100:00100000:3.0:1789762915.842308:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074730880:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.842333:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074731904:12345-0@lo:400:kworker.0 Request processed in 66us (758us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762915.842342:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 583 00000100:00100000:2.0:1789762915.842345:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 240 00000100:00100000:0.0:1789762915.842357:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 584 00000100:00100000:1.0:1789762915.842366:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074731392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.842380:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:0.0:1789762915.842381:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074731776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762915.842382:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074730880:12345-0@lo:400:kworker.0 Request processed in 81us (997us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762915.842397:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074732032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762915.842404:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 582 00000100:00100000:3.0:1789762915.842431:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 585 00000100:00100000:1.0:1789762915.842442:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074731392:12345-0@lo:400:kworker.0 Request processed in 82us (957us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762915.842444:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c8edce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074731136:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.842458:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074732032:12345-0@lo:400:kworker.0 Request processed in 66us (597us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762915.842465:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 583 00000100:00100000:0.0:1789762915.842480:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074731776:12345-0@lo:400:kworker.0 Request processed in 107us (944us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762915.842487:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00100000:3.0:1789762915.842501:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c8edce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074731136:12345-0@lo:400:kworker.0 Request processed in 60us (765us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762915.842503:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 584 00000100:00100000:2.0:1789762915.842513:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:1.0:1789762915.842519:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 586 00000100:00100000:2.0:1789762915.842527:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074732160:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762915.842533:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 585 00000100:00100000:1.0:1789762915.842541:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074731520:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.842548:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 587 00000100:00100000:0.0:1789762915.842566:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d69180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074731648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762915.842577:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074731264 00000100:00100000:3.0:1789762915.842583:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074731264 00000100:00100000:2.0:1789762915.842607:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074732160:12345-0@lo:400:kworker.0 Request processed in 82us (647us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762915.842620:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00100000:3.0:1789762915.842623:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074731008 00000100:00100000:1.0:1789762915.842626:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074731520:12345-0@lo:400:kworker.0 Request processed in 90us (852us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762915.842629:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074731008 00000100:00100000:1.0:1789762915.842642:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 586 00000100:00100000:0.0:1789762915.842649:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d69180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074731648:12345-0@lo:400:kworker.0 Request processed in 81us (813us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762915.842664:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 587 00000100:00100000:3.0:1789762915.842670:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:2.0:1789762915.842679:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 391 00000100:00100000:3.0:1789762915.842685:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074731264:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.842697:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074731008:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762915.842727:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074730752 00000100:00100000:1.0:1789762915.842735:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074730752 00000100:00100000:3.0:1789762915.842740:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074731264:12345-0@lo:400:kworker.0 Request processed in 59us (1309us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762915.842750:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00100000:2.0:1789762915.842755:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074731008:12345-0@lo:400:kworker.0 Request processed in 64us (1071us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762915.842768:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762915.842769:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 391 00000100:00100000:1.0:1789762915.842798:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 378 00000100:00100000:1.0:1789762915.842814:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb93bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074730752:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.842837:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074730624:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762915.842848:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074730496 00000100:00100000:0.0:1789762915.842857:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074730496 00000100:00100000:2.0:1789762915.842857:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb591180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074731136:0@lo:400:kworker.0 00000100:00100000:1.0:1789762915.842881:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb93bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074730752:12345-0@lo:400:kworker.0 Request processed in 68us (1531us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762915.842883:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb592a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074731648:0@lo:400:kworker.0 00000100:00100000:1.0:1789762915.842902:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 378 00000100:00100000:0.0:1789762915.842912:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:2.0:1789762915.842948:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb591c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074732160:0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.842978:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb592d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074730496:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762915.842997:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb590e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074731776:0@lo:400:kworker.0 00002000:00000024:0.0:1789762915.843023:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762915.843026:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eaed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074730752:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.843051:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb590a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074731264:0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.843101:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb592d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074730496:12345-0@lo:13:osp-pre-1-1.0 Request processed in 129us (336us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762915.843113:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00100000:2.0:1789762915.843128:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb593480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074731008:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.843146:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb590380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074731520:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.843163:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb591880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074732032:0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.843197:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eadc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074730496:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762915.843202:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0ead180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074730880:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.843226:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb593100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074731392:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.843243:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb591500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074731904:0@lo:400:kworker.0 00000100:00080000:0.0:1789762915.843267:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762915.843275:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1789762915.968486:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762916.992092:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762917.888196:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff614a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074732288:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762917.888313:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762917.888365:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074732288 00000100:00100000:1.0:1789762917.888376:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074732288 00000100:00100000:1.0:1789762917.888452:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:1.0:1789762917.888466:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074732288:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762917.888506:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762917.888605:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074732288:12345-0@lo:41:osp-pre-1-0.0 Request processed in 144us (295us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762917.888621:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00100000:3.0:1789762917.888716:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff614a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074732288:0@lo:41:osp-pre-1-0.0 00000001:00100000:3.0:1789762917.898219:0:9000:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000020:00000080:3.0:1789762917.898226:0:9000:0:(genops.c:885:class_export_put()) final put ffff9c37e1bee000/fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000020:00000080:0.0:1789762917.898254:0:38:0:(genops.c:844:class_export_destroy()) destroying export ffff9c37e1bee000/fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 for lustre-OST0001 00000020:00000080:0.0:1789762917.898309:0:38:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9c37e1bef800 for lustre-OST0001 00010000:00010000:3.0:1789762918.016095:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762918.912166:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074732416:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762918.912235:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762918.912270:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074732416 00000100:00100000:0.0:1789762918.912278:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074732416 00000100:00100000:0.0:1789762918.912340:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:0.0:1789762918.912352:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff617100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074732416:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762918.912400:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762918.912472:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff617100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074732416:12345-0@lo:41:osp-pre-0-1.0 Request processed in 125us (241us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762918.912479:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:1.0:1789762918.912526:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074732416:0@lo:41:osp-pre-0-1.0 00000001:00080000:1.0:1789762918.936722:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968031 is committed 00010000:00010000:0.0:1789762919.040040:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762919.880049:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762919.880081:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762919.880100:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762919.880117:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067515904 00000100:00100000:3.0:1789762919.880121:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067516032 00000100:00100000:0.0:1789762919.880125:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762919.880127:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067515904 00000100:00100000:3.0:1789762919.880129:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067516032 00000100:00100000:1.0:1789762919.880149:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067516160 00000100:00100000:0.0:1789762919.880155:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762919.880162:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067516160 00000100:00100000:2.0:1789762919.880185:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 379 00000100:00100000:3.0:1789762919.880188:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067516416 00000100:00100000:0.0:1789762919.880195:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762919.880195:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067516416 00000100:00100000:2.0:1789762919.880201:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb93b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067515904:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762919.880211:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067516544 00000100:00100000:1.0:1789762919.880216:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067516544 00000100:00080000:0.0:1789762919.880221:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762919.880227:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762919.880229:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 588 00000100:00100000:3.0:1789762919.880240:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb590700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067516032:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762919.880249:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 589 00000100:00100000:1.0:1789762919.880263:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 590 00000100:00100000:0.0:1789762919.880264:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb593800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067516160:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762919.880270:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb93b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067515904:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 73us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762919.880283:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 379 00000100:00100000:1.0:1789762919.880284:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb592300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067516416:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762919.880293:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb590700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067516032:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 55us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762919.880300:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 588 00000100:00100000:0.0:1789762919.880305:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb593800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067516160:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 46us (202us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762919.880313:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 589 00000100:00100000:2.0:1789762919.880327:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067516288 00000100:00100000:3.0:1789762919.880327:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 591 00000100:00100000:1.0:1789762919.880332:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb592300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067516416:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 54us (176us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762919.880335:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067516288 00000100:00100000:1.0:1789762919.880344:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 590 00000100:00100000:3.0:1789762919.880345:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb592680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067516544:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762919.880375:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb592680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067516544:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 38us (184us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762919.880380:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 591 00000100:00080000:1.0:1789762919.880393:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762919.880399:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 243 00000100:00100000:2.0:1789762919.880411:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb591f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067516288:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762919.880443:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb591f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067516288:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 35us (318us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762919.880450:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 243 00010000:00010000:1.0:1789762920.064037:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762920.960144:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074732544:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762920.960176:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d6b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074732672:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762920.960193:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff615500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074732800:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762920.960237:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762920.960268:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb590000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074732928:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762920.960287:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074732544 00000100:00100000:1.0:1789762920.960300:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074732544 00000100:00100000:2.0:1789762920.960322:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762920.960335:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb593b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074733056:0@lo:400:kworker.0 00000100:00100000:1.0:1789762920.960390:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074732928 00000100:00100000:3.0:1789762920.960391:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762920.960400:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074732928 00000100:00100000:2.0:1789762920.960405:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb593100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074733312:0@lo:400:kworker.0 00000100:00100000:0.0:1789762920.960414:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:0.0:1789762920.960432:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff698a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074732544:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762920.960445:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb590380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074733568:0@lo:400:kworker.0 00000100:00100000:1.0:1789762920.960460:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 243 00000100:00100000:2.0:1789762920.960467:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:0.0:1789762920.960477:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762920.960479:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074732928:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789762920.960502:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762920.960509:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074733056 00000100:00100000:2.0:1789762920.960512:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb590a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074733824:0@lo:400:kworker.0 00000100:00100000:3.0:1789762920.960516:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074733056 00000100:00100000:0.0:1789762920.960534:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff698a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074732544:12345-0@lo:13:osp-pre-1-0.0 Request processed in 107us (298us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762920.960542:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00100000:3.0:1789762920.960583:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:1.0:1789762920.960594:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074732928:12345-0@lo:13:osp-pre-1-1.0 Request processed in 113us (269us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762920.960597:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762920.960599:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762920.960605:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb93a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074733056:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762920.960624:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 243 00000100:00100000:0.0:1789762920.960635:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074733312 00000100:00100000:0.0:1789762920.960641:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074733312 00000100:00100000:2.0:1789762920.960650:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb591180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074734336:0@lo:400:kworker.0 00000100:00100000:3.0:1789762920.960668:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb93a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074733056:12345-0@lo:400:kworker.0 Request processed in 71us (281us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762920.960681:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00100000:1.0:1789762920.960688:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762920.960690:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:2.0:1789762920.960701:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762920.960714:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074733312:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762920.960751:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762920.960785:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074732544:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762920.960789:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb590000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074732928:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762920.960796:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074733312:12345-0@lo:400:kworker.0 Request processed in 82us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762920.960796:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb591880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074733440:0@lo:400:kworker.0 00000100:00100000:0.0:1789762920.960818:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1789762920.960833:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb591c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074734080:0@lo:400:kworker.0 00000100:00100000:0.0:1789762920.960840:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074733568 00000100:00100000:0.0:1789762920.960844:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074733568 00000100:00100000:3.0:1789762920.960855:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762920.960856:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb593100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074733312:0@lo:400:kworker.0 00000100:00100000:2.0:1789762920.960882:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1789762920.960891:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:3.0:1789762920.960899:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb590e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074733952:0@lo:400:kworker.0 00000100:00100000:2.0:1789762920.960903:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb591500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074733184:0@lo:400:kworker.0 00000100:00100000:1.0:1789762920.960904:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762920.960913:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074733568:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762920.960998:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb593b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074733056:0@lo:400:kworker.0 00000100:00100000:3.0:1789762920.961012:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762920.961028:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074733568:12345-0@lo:400:kworker.0 Request processed in 123us (349us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762920.961033:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb590380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074733568:0@lo:400:kworker.0 00000100:00100000:2.0:1789762920.961035:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762920.961050:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:3.0:1789762920.961061:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074733824 00000100:00100000:3.0:1789762920.961067:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074733824 00000100:00100000:0.0:1789762920.961101:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074733440 00000100:00100000:0.0:1789762920.961108:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074733440 00000100:00100000:1.0:1789762920.961109:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074734080 00000100:00100000:1.0:1789762920.961120:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074734080 00000100:00100000:2.0:1789762920.961121:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762920.961127:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074733952 00000100:00100000:3.0:1789762920.961131:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074733952 00000100:00100000:0.0:1789762920.961145:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074733184 00000100:00100000:0.0:1789762920.961150:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074733184 00000100:00100000:2.0:1789762920.961161:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb593480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074733696:0@lo:400:kworker.0 00000100:00100000:1.0:1789762920.961191:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 592 00000100:00100000:3.0:1789762920.961198:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 593 00000100:00100000:0.0:1789762920.961212:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 594 00000100:00100000:1.0:1789762920.961212:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d68e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074733824:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762920.961214:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074733440:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762920.961223:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762920.961236:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074734080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762920.961252:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb592a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074734208:0@lo:400:kworker.0 00000100:00100000:3.0:1789762920.961309:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074733440:12345-0@lo:400:kworker.0 Request processed in 101us (456us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762920.961314:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762920.961321:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 593 00000100:00100000:0.0:1789762920.961328:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074734080:12345-0@lo:400:kworker.0 Request processed in 95us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762920.961332:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d68e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074733824:12345-0@lo:400:kworker.0 Request processed in 124us (738us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762920.961344:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074733696 00000100:00100000:0.0:1789762920.961352:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 594 00000100:00100000:3.0:1789762920.961355:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074733696 00000100:00100000:1.0:1789762920.961361:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 592 00000100:00100000:2.0:1789762920.961366:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb590a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074733824:0@lo:400:kworker.0 00000100:00100000:0.0:1789762920.961386:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 595 00000100:00100000:0.0:1789762920.961401:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074733952:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762920.961413:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 596 00000100:00100000:2.0:1789762920.961425:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074734336 00000100:00100000:3.0:1789762920.961425:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 597 00000100:00100000:1.0:1789762920.961432:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d69c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074733184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762920.961433:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074734336 00000100:00100000:0.0:1789762920.961444:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074733952:12345-0@lo:400:kworker.0 Request processed in 48us (437us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762920.961444:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074733696:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762920.961457:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 595 00000100:00100000:0.0:1789762920.961512:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074734208 00000100:00100000:1.0:1789762920.961518:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d69c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074733184:12345-0@lo:400:kworker.0 Request processed in 88us (400us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762920.961521:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074734208 00000100:00100000:2.0:1789762920.961521:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 244 00000100:00100000:3.0:1789762920.961528:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074733696:12345-0@lo:400:kworker.0 Request processed in 87us (321us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762920.961548:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d69500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074734336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762920.961550:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 596 00000100:00100000:0.0:1789762920.961562:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:3.0:1789762920.961564:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 597 00000100:00100000:0.0:1789762920.961578:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074734208:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762920.961616:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb591c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074734080:0@lo:400:kworker.0 00000100:00100000:3.0:1789762920.961622:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074732800 00000100:00100000:2.0:1789762920.961624:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d69500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074734336:12345-0@lo:400:kworker.0 Request processed in 83us (929us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762920.961631:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074732800 00000100:00100000:0.0:1789762920.961638:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074734208:12345-0@lo:400:kworker.0 Request processed in 63us (328us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762920.961646:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 244 00000100:00100000:0.0:1789762920.961654:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00100000:3.0:1789762920.961665:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074732672 00000100:00100000:3.0:1789762920.961669:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074732672 00000100:00100000:1.0:1789762920.961698:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 244 00000100:00100000:0.0:1789762920.961710:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:1.0:1789762920.961715:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074732800:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762920.961715:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb591500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074733184:0@lo:400:kworker.0 00000100:00080000:3.0:1789762920.961734:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762920.961735:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074732672:12345-0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1789762920.961741:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762920.961746:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb593480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074733696:0@lo:400:kworker.0 00000100:00080000:3.0:1789762920.961748:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00002000:00000024:1.0:1789762920.961750:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00002000:00000024:0.0:1789762920.961752:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762920.961772:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb591880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074733440:0@lo:400:kworker.0 00000100:00100000:2.0:1789762920.961801:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb592a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074734208:0@lo:400:kworker.0 00000100:00100000:3.0:1789762920.961801:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb590e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074733952:0@lo:400:kworker.0 00000100:00100000:2.0:1789762920.961833:0:3425:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294968031 gen 2 00000004:00080000:2.0:1789762920.961839:0:3425:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37ff616680, transno 4294968031 00000100:00100000:1.0:1789762920.961851:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074732800:12345-0@lo:13:osp-pre-0-0.0 Request processed in 143us (1102us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762920.961857:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074732672:12345-0@lo:13:osp-pre-0-1.0 Request processed in 126us (825us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762920.961864:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb591180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074734336:0@lo:400:kworker.0 00000100:00100000:3.0:1789762920.961866:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff615500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074732800:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762920.961886:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 244 00000100:00100000:0.0:1789762920.961897:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00100000:0.0:1789762920.961985:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d6b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074732672:0@lo:13:osp-pre-0-1.0 00010000:00010000:0.0:1789762921.088101:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762922.112057:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762923.008099:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789762923.008122:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff616680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074734464:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762923.008200:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762923.008251:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074734464 00000100:00100000:1.0:1789762923.008260:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074734464 00000100:00100000:1.0:1789762923.008309:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:1.0:1789762923.008321:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074734464:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762923.008354:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762923.008410:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074734464:12345-0@lo:41:osp-pre-1-0.0 Request processed in 93us (210us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762923.008417:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:3.0:1789762923.008448:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff616680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074734464:0@lo:41:osp-pre-1-0.0 00010000:00010000:0.0:1789762923.136162:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762924.032133:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074734592:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762924.032216:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762924.032273:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074734592 00000100:00100000:2.0:1789762924.032283:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074734592 00000100:00100000:2.0:1789762924.032349:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:2.0:1789762924.032360:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074734592:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762924.032405:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762924.032460:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074734592:12345-0@lo:41:osp-pre-0-1.0 Request processed in 101us (243us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762924.032466:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:1.0:1789762924.032506:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074734592:0@lo:41:osp-pre-0-1.0 00010000:00010000:1.0:1789762924.160083:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762925.000109:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762925.000154:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762925.000177:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762925.000196:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067516800 00000100:00100000:2.0:1789762925.000213:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067516800 00000100:00100000:1.0:1789762925.000215:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067516672 00000100:00100000:0.0:1789762925.000224:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762925.000224:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067516672 00000100:00100000:3.0:1789762925.000255:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067517056 00000100:00100000:3.0:1789762925.000267:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067517056 00000100:00100000:0.0:1789762925.000286:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762925.000324:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 381 00000100:00100000:2.0:1789762925.000325:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067516928 00000100:00100000:2.0:1789762925.000330:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067516928 00000100:00100000:1.0:1789762925.000342:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb93b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067516672:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762925.000361:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762925.000367:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 246 00000100:00100000:2.0:1789762925.000373:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067517312 00000100:00100000:2.0:1789762925.000378:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067517312 00000100:00100000:3.0:1789762925.000388:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb592a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067517056:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762925.000401:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067517184 00000100:00100000:0.0:1789762925.000408:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067517184 00000100:00100000:2.0:1789762925.000412:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 598 00000100:00100000:2.0:1789762925.000425:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb591180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067516800:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762925.000437:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb93b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067516672:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 99us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762925.000455:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 381 00000100:00100000:0.0:1789762925.000456:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 599 00000100:00100000:3.0:1789762925.000469:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb592a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067517056:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 84us (285us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762925.000483:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb591180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067516800:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 54us (373us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762925.000487:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 246 00000100:00100000:0.0:1789762925.000489:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb593480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067516928:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762925.000517:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 598 00000100:00100000:2.0:1789762925.000564:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 600 00000100:00100000:0.0:1789762925.000565:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb593480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067516928:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 84us (340us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762925.000578:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 601 00000100:00100000:2.0:1789762925.000579:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb591500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067517312:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762925.000595:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 599 00000100:00100000:1.0:1789762925.000600:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb590e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067517184:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:3.0:1789762925.000608:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762925.000620:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789762925.000626:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762925.000632:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb591500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067517312:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 50us (341us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762925.000640:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 600 00000100:00100000:1.0:1789762925.000665:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb590e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067517184:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 73us (302us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762925.000680:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 601 00010000:00010000:0.0:1789762925.184137:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762926.080101:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074734976:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762926.080142:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb591880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074735104:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762926.080157:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f58700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074735360:0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.080192:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762926.080199:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074735232:0@lo:400:kworker.0 00000100:00100000:0.0:1789762926.080258:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762926.080263:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762926.080279:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff614a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074734848:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762926.080322:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762926.080327:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074735872:0@lo:400:kworker.0 00000100:00100000:2.0:1789762926.080338:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074735616:0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.080347:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074734976 00000100:00100000:1.0:1789762926.080356:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074734976 00000100:00100000:3.0:1789762926.080372:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f58380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074735744:0@lo:400:kworker.0 00000100:00100000:0.0:1789762926.080385:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762926.080400:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074735232 00000100:00100000:2.0:1789762926.080406:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074735232 00000100:00100000:1.0:1789762926.080435:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074735104 00000100:00100000:0.0:1789762926.080436:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074736384:0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.080437:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762926.080445:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074735104 00000100:00100000:2.0:1789762926.080458:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:3.0:1789762926.080476:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e81180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074736256:0@lo:400:kworker.0 00000100:00100000:2.0:1789762926.080482:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb939850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074735232:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.080501:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:0.0:1789762926.080505:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762926.080514:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074734976:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762926.080521:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:1.0:1789762926.080560:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762926.080572:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 247 00000100:00100000:2.0:1789762926.080578:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb939850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074735232:12345-0@lo:400:kworker.0 Request processed in 109us (320us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762926.080587:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074735104:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762926.080589:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:3.0:1789762926.080606:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762926.080623:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074734976:12345-0@lo:13:osp-pre-0-1.0 Request processed in 109us (429us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762926.080632:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00002000:00000024:0.0:1789762926.080635:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762926.080659:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074736128:0@lo:400:kworker.0 00000100:00100000:2.0:1789762926.080661:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074735744 00000100:00100000:2.0:1789762926.080667:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074735744 00000100:00080000:1.0:1789762926.080668:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789762926.080678:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762926.080706:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074735104:12345-0@lo:13:osp-pre-1-1.0 Request processed in 119us (388us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762926.080706:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762926.080719:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:3.0:1789762926.080730:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762926.080750:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 247 00000100:00100000:2.0:1789762926.080750:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e80e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074735744:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.080751:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074734720:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762926.080758:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074736640:0@lo:400:kworker.0 00000100:00100000:0.0:1789762926.080800:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074734848 00000100:00100000:0.0:1789762926.080807:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074734848 00000100:00100000:1.0:1789762926.080821:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762926.080825:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762926.080831:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e80e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074735744:12345-0@lo:400:kworker.0 Request processed in 96us (397us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762926.080839:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:1.0:1789762926.080862:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074735488:0@lo:400:kworker.0 00000100:00100000:2.0:1789762926.080873:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074736384 00000100:00100000:3.0:1789762926.080876:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074735232:0@lo:400:kworker.0 00000100:00100000:2.0:1789762926.080878:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074736384 00000100:00100000:0.0:1789762926.080885:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074734720 00000100:00100000:0.0:1789762926.080893:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074734720 00000100:00100000:1.0:1789762926.080894:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762926.080962:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 247 00000100:00100000:1.0:1789762926.080968:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e83b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074736000:0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.080969:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074735360 00000100:00100000:3.0:1789762926.080977:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074735360 00000100:00100000:2.0:1789762926.080981:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074736384:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762926.081000:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 248 00000100:00100000:1.0:1789762926.081004:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762926.081015:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff616680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074734848:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762926.081030:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074735872 00002000:00000024:0.0:1789762926.081032:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762926.081034:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074735872 00000100:00100000:1.0:1789762926.081036:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e83100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074736512:0@lo:400:kworker.0 00000100:00100000:2.0:1789762926.081046:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074736384:12345-0@lo:400:kworker.0 Request processed in 69us (549us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762926.081054:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 247 00000100:00100000:3.0:1789762926.081070:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074736256 00000100:00100000:2.0:1789762926.081074:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074736640 00000100:00100000:3.0:1789762926.081074:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074736256 00000100:00100000:2.0:1789762926.081080:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074736640 00000100:00100000:1.0:1789762926.081116:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762926.081120:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074735616 00000100:00100000:0.0:1789762926.081121:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff616680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074734848:12345-0@lo:13:osp-pre-0-0.0 Request processed in 108us (417us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762926.081127:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074735616 00000100:00100000:2.0:1789762926.081128:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074736512 00000100:00100000:0.0:1789762926.081137:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 248 00000100:00100000:2.0:1789762926.081138:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074736512 00000100:00100000:3.0:1789762926.081184:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074736128 00000100:00100000:1.0:1789762926.081185:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074734976:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762926.081194:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 249 00000100:00100000:3.0:1789762926.081194:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074736128 00000100:00100000:2.0:1789762926.081195:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 248 00000100:00100000:1.0:1789762926.081223:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff614a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074734848:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762926.081223:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074736000 00000100:00100000:3.0:1789762926.081228:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074736000 00000100:00100000:0.0:1789762926.081246:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3424:x1876702074734720:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762926.081247:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074736640:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.081264:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 602 00000100:00100000:1.0:1789762926.081275:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074735488 00000100:00100000:1.0:1789762926.081281:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074735488 00000100:00100000:3.0:1789762926.081281:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb591c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074735360:12345-0@lo:400:kworker.0 00002000:00000024:0.0:1789762926.081288:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762926.081306:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074736640:12345-0@lo:400:kworker.0 Request processed in 84us (484us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762926.081313:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 248 00000100:00100000:1.0:1789762926.081325:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:2.0:1789762926.081332:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 249 00000100:00100000:1.0:1789762926.081336:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074735488:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762926.081347:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074736512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.081357:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb591c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074735360:12345-0@lo:400:kworker.0 Request processed in 79us (1105us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762926.081367:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 602 00000100:00100000:0.0:1789762926.081375:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074734720:12345-0@lo:13:osp-pre-1-0.0 Request processed in 152us (557us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762926.081388:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 249 00000100:00100000:3.0:1789762926.081391:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 603 00000100:00100000:3.0:1789762926.081405:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb590a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074735872:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.081413:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074735488:12345-0@lo:400:kworker.0 Request processed in 73us (512us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762926.081413:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074736512:12345-0@lo:400:kworker.0 Request processed in 67us (303us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762926.081427:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 249 00000100:00100000:1.0:1789762926.081440:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:0.0:1789762926.081445:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 604 00000100:00100000:3.0:1789762926.081454:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb590a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074735872:12345-0@lo:400:kworker.0 Request processed in 54us (1074us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762926.081465:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e81500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074736256:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.081475:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 603 00000100:00100000:2.0:1789762926.081488:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 605 00000100:00100000:1.0:1789762926.081500:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 606 00000100:00100000:2.0:1789762926.081501:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e81880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074735616:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.081516:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 607 00000100:00100000:1.0:1789762926.081521:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e81c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074736128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.081538:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fd9ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074736000:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762926.081539:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e81500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074736256:12345-0@lo:400:kworker.0 Request processed in 75us (1020us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762926.081558:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e81880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074735616:12345-0@lo:400:kworker.0 Request processed in 57us (953us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762926.081575:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e81c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074736128:12345-0@lo:400:kworker.0 Request processed in 56us (851us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762926.081576:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 604 00000100:00100000:3.0:1789762926.081587:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fd9ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074736000:12345-0@lo:400:kworker.0 Request processed in 61us (585us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762926.081592:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 605 00000100:00100000:1.0:1789762926.081603:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 606 00000100:00100000:3.0:1789762926.081613:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 607 00000100:00080000:0.0:1789762926.081615:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762926.081625:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762926.081633:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762926.081639:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762926.081664:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074735872:0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.081665:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074735488:0@lo:400:kworker.0 00000100:00100000:0.0:1789762926.081701:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f58380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074735744:0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.081707:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074736128:0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.081708:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e83b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074736000:0@lo:400:kworker.0 00000100:00100000:2.0:1789762926.081713:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074736384:0@lo:400:kworker.0 00000100:00100000:0.0:1789762926.081730:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e81180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074736256:0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.081744:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e83100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074736512:0@lo:400:kworker.0 00000100:00100000:2.0:1789762926.081759:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f58700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074735360:0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.081761:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074736640:0@lo:400:kworker.0 00000100:00100000:0.0:1789762926.081780:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb591880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074735104:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762926.081794:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074734720:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762926.081799:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074735616:0@lo:400:kworker.0 00010000:00010000:0.0:1789762926.208071:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762927.232064:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762928.128108:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789762928.128128:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074736768:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762928.128190:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762928.128235:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074736768 00000100:00100000:2.0:1789762928.128245:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074736768 00000100:00100000:2.0:1789762928.128297:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:2.0:1789762928.128310:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074736768:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762928.128350:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762928.128407:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074736768:12345-0@lo:41:osp-pre-1-0.0 Request processed in 100us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762928.128413:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:1.0:1789762928.128436:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074736768:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1789762928.256075:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762929.152110:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074736896:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762929.152214:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762929.152356:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074736896 00000100:00100000:2.0:1789762929.152366:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074736896 00000100:00100000:2.0:1789762929.152426:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:2.0:1789762929.152437:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074736896:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762929.152527:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:2.0:1789762929.152663:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074736896:12345-0@lo:41:osp-pre-0-1.0 Request processed in 206us (432us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762929.152672:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:1.0:1789762929.152754:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074736896:0@lo:41:osp-pre-0-1.0 00010000:00010000:0.0:1789762929.280119:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762930.120018:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762930.120057:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762930.120080:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067517568 00000100:00100000:0.0:1789762930.120085:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762930.120091:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067517568 00000100:00100000:0.0:1789762930.120113:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762930.120145:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067517440 00000100:00100000:3.0:1789762930.120149:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067517824 00000100:00100000:0.0:1789762930.120157:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762930.120158:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067517824 00000100:00100000:1.0:1789762930.120161:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067517440 00000100:00100000:2.0:1789762930.120173:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067517696 00000100:00100000:2.0:1789762930.120178:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067517696 00000100:00100000:3.0:1789762930.120223:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 250 00000100:00100000:2.0:1789762930.120224:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067518080 00000100:00100000:2.0:1789762930.120230:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067518080 00000100:00100000:3.0:1789762930.120241:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067517824:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762930.120254:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 383 00000100:00100000:0.0:1789762930.120262:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762930.120269:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067517952 00000100:00100000:1.0:1789762930.120274:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37eb938c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067517440:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762930.120276:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067517952 00000100:00100000:0.0:1789762930.120295:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 608 00000100:00100000:0.0:1789762930.120309:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb591880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067517568:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762930.120319:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 609 00000100:00100000:3.0:1789762930.120330:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067517824:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 94us (241us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762930.120332:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb593100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067517696:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762930.120358:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37eb938c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2397:x1876702067517440:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 87us (291us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762930.120358:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 250 00000100:00100000:0.0:1789762930.120381:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb591880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067517568:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 67us (361us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762930.120392:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 383 00000100:00100000:2.0:1789762930.120399:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb593100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067517696:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 55us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762930.120405:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 608 00000100:00100000:2.0:1789762930.120420:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 609 00000100:00100000:3.0:1789762930.120433:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 610 00000100:00100000:3.0:1789762930.120449:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb590000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067518080:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762930.120454:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 611 00000100:00100000:0.0:1789762930.120463:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e71880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067517952:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762930.120499:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb590000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067518080:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 53us (346us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762930.120507:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e71880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067517952:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 45us (248us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762930.120512:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762930.120513:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 610 00000100:00080000:1.0:1789762930.120530:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762930.120531:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 611 00000100:00080000:1.0:1789762930.120543:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1789762930.304144:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762931.200083:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074737408:0@lo:400:kworker.0 00000100:00100000:2.0:1789762931.200115:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d6a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074737664:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762931.200118:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e83800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074737792:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.200197:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762931.200198:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762931.200199:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762931.200213:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074737536:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.200268:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9e71c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074737152:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762931.200273:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762931.200284:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074738304:0@lo:400:kworker.0 00000100:00100000:2.0:1789762931.200284:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e82680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074738176:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.200309:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762931.200312:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e82300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074738048:0@lo:400:kworker.0 00000100:00100000:2.0:1789762931.200336:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762931.200340:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074737024:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762931.200346:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074737792 00000100:00100000:1.0:1789762931.200355:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074737792 00000100:00100000:3.0:1789762931.200362:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762931.200369:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e83b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074738688:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.200377:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762931.200399:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e83100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074738560:0@lo:400:kworker.0 00000100:00100000:1.0:1789762931.200405:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074738048 00000100:00100000:0.0:1789762931.200409:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074737280:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762931.200411:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074738048 00000100:00100000:2.0:1789762931.200414:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762931.200446:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762931.200453:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:0.0:1789762931.200459:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762931.200470:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074737792:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762931.200476:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1789762931.200488:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074739072:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.200492:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e81f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074737920:0@lo:400:kworker.0 00000100:00100000:2.0:1789762931.200494:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074738048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762931.200552:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762931.200553:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762931.200554:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074737792:12345-0@lo:400:kworker.0 Request processed in 90us (361us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762931.200562:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074738048:12345-0@lo:400:kworker.0 Request processed in 74us (201us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762931.200568:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:2.0:1789762931.200577:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:0.0:1789762931.200586:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074738432:0@lo:400:kworker.0 00000100:00100000:1.0:1789762931.200604:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074737664 00000100:00100000:1.0:1789762931.200610:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074737664 00000100:00100000:2.0:1789762931.200611:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074738688 00000100:00100000:2.0:1789762931.200617:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074738688 00000100:00100000:3.0:1789762931.200623:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e82300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074738048:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.200624:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762931.200652:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074738944:0@lo:400:kworker.0 00000100:00100000:2.0:1789762931.200669:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074739072 00000100:00100000:2.0:1789762931.200673:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074739072 00000100:00100000:1.0:1789762931.200682:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074737152 00000100:00100000:0.0:1789762931.200683:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762931.200686:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074737024 00000100:00100000:1.0:1789762931.200689:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074737152 00000100:00100000:3.0:1789762931.200693:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074737024 00000100:00100000:2.0:1789762931.200705:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074738944 00000100:00100000:2.0:1789762931.200710:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074738944 00000100:00100000:1.0:1789762931.200723:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074737280 00000100:00100000:1.0:1789762931.200730:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074737280 00000100:00100000:3.0:1789762931.200735:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 250 00000100:00100000:2.0:1789762931.200743:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 251 00000100:00100000:3.0:1789762931.200751:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074737664:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762931.200757:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074738688:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.200762:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:1.0:1789762931.200767:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 251 00000100:00100000:1.0:1789762931.200782:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e72d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074737152:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762931.200785:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074739072:12345-0@lo:400:kworker.0 00002000:00000024:3.0:1789762931.200797:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762931.200799:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074738688:12345-0@lo:400:kworker.0 Request processed in 49us (391us total) trans 0 rc 0/0 00002000:00000024:1.0:1789762931.200803:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762931.200809:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 251 00000100:00100000:2.0:1789762931.200835:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 253 00000100:00100000:0.0:1789762931.200845:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074739072:12345-0@lo:400:kworker.0 Request processed in 71us (301us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762931.200849:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e72680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3424:x1876702074738944:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.200858:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00100000:1.0:1789762931.200881:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e72d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074737152:12345-0@lo:13:osp-pre-1-1.0 Request processed in 92us (559us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762931.200884:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074737664:12345-0@lo:13:osp-pre-1-0.0 Request processed in 121us (674us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762931.200892:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 251 00000100:00100000:3.0:1789762931.200901:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 250 00000100:00100000:2.0:1789762931.200942:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e72680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3424:x1876702074738944:12345-0@lo:400:kworker.0 Request processed in 76us (239us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762931.200943:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:1.0:1789762931.200958:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 253 00000100:00100000:2.0:1789762931.200958:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 253 00000100:00100000:0.0:1789762931.200959:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e70700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074737024:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762931.200975:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e73800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074737280:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762931.200985:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00002000:00000024:1.0:1789762931.200987:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00080000:3.0:1789762931.201001:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789762931.201009:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762931.201018:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e83b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074738688:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.201037:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e70700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074737024:12345-0@lo:13:osp-pre-0-1.0 Request processed in 80us (660us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762931.201038:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074737536 00000100:00100000:1.0:1789762931.201046:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e73800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074737280:12345-0@lo:13:osp-pre-0-0.0 Request processed in 72us (585us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762931.201046:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d6a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074737664:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762931.201047:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074737536 00000100:00100000:0.0:1789762931.201052:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00100000:1.0:1789762931.201064:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 253 00000100:00100000:3.0:1789762931.201102:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074738176 00000100:00100000:0.0:1789762931.201107:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074737920 00000100:00100000:1.0:1789762931.201107:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074738560 00000100:00100000:2.0:1789762931.201108:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074738432 00000100:00100000:3.0:1789762931.201109:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074738176 00000100:00100000:1.0:1789762931.201113:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074738560 00000100:00100000:0.0:1789762931.201115:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074737920 00000100:00100000:2.0:1789762931.201116:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074738432 00000100:00100000:0.0:1789762931.201153:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 612 00000100:00100000:0.0:1789762931.201166:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074737536:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762931.201167:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 613 00000100:00100000:1.0:1789762931.201178:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 614 00000100:00100000:3.0:1789762931.201183:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e71f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074737920:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762931.201186:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 615 00000100:00100000:1.0:1789762931.201191:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d68000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074738176:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.201206:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074737536:12345-0@lo:400:kworker.0 Request processed in 41us (934us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762931.201206:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074738560:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.201217:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 612 00000100:00100000:0.0:1789762931.201237:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 616 00000100:00100000:0.0:1789762931.201248:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e72300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074738432:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762931.201255:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e71f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074737920:12345-0@lo:400:kworker.0 Request processed in 73us (709us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762931.201263:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d68000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074738176:12345-0@lo:400:kworker.0 Request processed in 76us (926us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762931.201267:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074738560:12345-0@lo:400:kworker.0 Request processed in 68us (822us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762931.201278:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 613 00000100:00100000:0.0:1789762931.201283:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e72300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074738432:12345-0@lo:400:kworker.0 Request processed in 38us (660us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762931.201289:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 614 00000100:00100000:2.0:1789762931.201297:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 615 00000100:00100000:0.0:1789762931.201303:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 616 00000100:00080000:3.0:1789762931.201310:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762931.201317:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762931.201322:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762931.201340:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e82680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074738176:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.201341:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074738304 00000100:00100000:0.0:1789762931.201347:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074738304 00000100:00100000:1.0:1789762931.201352:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e81180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074738816:0@lo:400:kworker.0 00000100:00100000:1.0:1789762931.201377:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762931.201395:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e81f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074737920:0@lo:400:kworker.0 00000100:00080000:3.0:1789762931.201396:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762931.201403:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 617 00000100:00100000:1.0:1789762931.201414:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e83800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074737792:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.201415:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074738304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762931.201418:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074737408 00000100:00100000:2.0:1789762931.201422:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074738432:0@lo:400:kworker.0 00000100:00100000:3.0:1789762931.201427:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074737408 00000100:00100000:2.0:1789762931.201442:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074738944:0@lo:400:kworker.0 00000100:00100000:2.0:1789762931.201462:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9e71c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074737152:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762931.201482:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074738304:12345-0@lo:400:kworker.0 Request processed in 69us (161us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762931.201490:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074737024:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762931.201490:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:1.0:1789762931.201492:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074738304:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.201495:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 617 00000100:00100000:3.0:1789762931.201510:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15c7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074737408:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.201530:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074738816 00000100:00100000:2.0:1789762931.201533:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074737280:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762931.201535:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074738816 00000100:00100000:3.0:1789762931.201568:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15c7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074737408:12345-0@lo:400:kworker.0 Request processed in 61us (1375us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762931.201577:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 254 00000100:00100000:3.0:1789762931.201582:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:1.0:1789762931.201587:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074737536:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.201592:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074738816:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762931.201597:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074737408:0@lo:400:kworker.0 00000100:00100000:1.0:1789762931.201613:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e83100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074738560:0@lo:400:kworker.0 00000100:00100000:1.0:1789762931.201636:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074739072:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.201642:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074738816:12345-0@lo:400:kworker.0 Request processed in 52us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762931.201654:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 254 00000100:00100000:3.0:1789762931.201676:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e81180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074738816:0@lo:400:kworker.0 00010000:00010000:0.0:1789762931.328037:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762932.352035:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762933.248109:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789762933.248136:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074739200:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762933.248221:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762933.248295:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074739200 00000100:00100000:2.0:1789762933.248312:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074739200 00000100:00100000:2.0:1789762933.248395:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:2.0:1789762933.248412:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074739200:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762933.248460:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762933.248537:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074739200:12345-0@lo:41:osp-pre-1-0.0 Request processed in 131us (316us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762933.248550:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00100000:1.0:1789762933.248601:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074739200:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762933.319043:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762933.319109:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067518208 00000100:00100000:1.0:1789762933.319123:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067518208 00000100:00080000:3.0:1789762933.319202:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762933.319212:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 338 00000100:00100000:1.0:1789762933.319229:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e70000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:15685:x1876702067518208:12345-192.168.203.37@tcp:101:mkdir.0 00010000:00010000:1.0:1789762933.319252:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762933.319276:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fa00/0xdf5a7ac59f17723f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca7e expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.319348:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177246 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.319365:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177246 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.319380:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177246 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762933.319420:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762933.319424:0:13389:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:1.0:1789762933.319463:0:13389:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177246 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.319496:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fa00/0xdf5a7ac59f17723f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca7e expref: 7 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.319544:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177246 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca7e expref: 6 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.319556:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762933.319561:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762933.319565:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d998f400, rc 0) 00000100:00100000:1.0:1789762933.319645:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e70000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15685:x1876702067518208:12345-192.168.203.37@tcp:101:mkdir.0 Request processed in 420us (609us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762933.319656:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 338 00000100:00100000:0.0:1789762933.321588:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762933.321649:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067518336 00000100:00100000:3.0:1789762933.321678:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067518336 00000100:00100000:0.0:1789762933.321695:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762933.321753:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067518592 00000100:00100000:3.0:1789762933.321760:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067518592 00000100:00100000:1.0:1789762933.321768:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 339 00000100:00100000:1.0:1789762933.321786:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e73b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067518336:12345-192.168.203.37@tcp:41:mkdir.0 00000100:00080000:0.0:1789762933.321816:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000004:00000024:1.0:1789762933.321829:0:6420:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762933.321886:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 340 00000100:00100000:0.0:1789762933.321912:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e73480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15685:x1876702067518592:12345-192.168.203.37@tcp:36:mkdir.0 00000100:00100000:1.0:1789762933.321945:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e73b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067518336:12345-192.168.203.37@tcp:41:mkdir.0 Request processed in 134us (329us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762933.321964:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 339 00000100:00100000:0.0:1789762933.322069:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9e71180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074739328:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762933.322124:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762933.322185:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f240 going to sleep for 11 seconds 00000100:00100000:1.0:1789762933.322194:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074739328 00000100:00100000:1.0:1789762933.322205:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074739328 00000100:00100000:1.0:1789762933.322262:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:1.0:1789762933.322273:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e70a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074739328:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762933.322366:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e70a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074739328:12345-0@lo:1000:mdt00_004.0 Request processed in 94us (240us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762933.322375:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:0.0:1789762933.322431:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9e71180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074739328:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762933.322527:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9e71180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074739456:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789762933.322565:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762933.322603:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f240 going to sleep for 11 seconds 00000100:00100000:1.0:1789762933.322622:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074739456 00000100:00100000:1.0:1789762933.322630:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074739456 00000100:00100000:1.0:1789762933.322674:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:1.0:1789762933.322689:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e73100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074739456:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762933.323105:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e73100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074739456:12345-0@lo:1000:mdt00_004.0 Request processed in 420us (538us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789762933.323118:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:2.0:1789762933.323147:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9e71180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074739456:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1789762933.323222:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.323240:0:11692:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.323258:0:11692:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762933.323278:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074739584:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1789762933.323322:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762933.323374:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff66e300 going to sleep for 11 seconds 00000100:00100000:1.0:1789762933.323379:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074739584 00000100:00100000:1.0:1789762933.323387:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074739584 00000100:00100000:1.0:1789762933.323453:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:1.0:1789762933.323465:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:11692:x1876702074739584:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1789762933.323479:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762933.323497:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f800/0xdf5a7ac59f177254 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f17724d expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.323526:0:6424:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177246 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca7e expref: 5 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.323540:0:6424:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f800/0xdf5a7ac59f177254 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f17724d expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.323575:0:6424:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177246 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca7e expref: 5 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.323593:0:6424:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177246 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca7e expref: 5 pid: 13389 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.323609:0:6424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962f800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074739712:192.168.203.37@tcp:104: 00000100:00100000:1.0:1789762933.323690:0:6424:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37eebc60c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762933.325108:0:6424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962f800 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074739712:192.168.203.37@tcp:104: 00010000:00010000:1.0:1789762933.325137:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f800/0xdf5a7ac59f177254 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17724d expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.325149:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762933.325152:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762933.325158:0:6424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f800/0xdf5a7ac59f177254 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17724d expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.325168:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d998f800, rc 0) 00000100:00100000:1.0:1789762933.325224:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:11692:x1876702074739584:12345-0@lo:101:mdt00_004.0 Request processed in 1761us (1902us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762933.325233:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00100000:2.0:1789762933.325275:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074739584:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1789762933.325296:0:11692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.325309:0:11692:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762933.325472:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762933.325522:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067518720 00000100:00100000:1.0:1789762933.325530:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067518720 00010000:00010000:1.0:1789762933.325557:0:6404:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177246 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca7e expref: 6 pid: 13389 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.325595:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 88 00000100:00100000:1.0:1789762933.325606:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067518720:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762933.325619:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762933.325626:0:6404:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177246 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca7e expref: 6 pid: 13389 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.325639:0:6404:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177246 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca7e expref: 6 pid: 13389 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.325659:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177246 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca7e expref: 6 pid: 13389 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.325674:0:6404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762933.325680:0:6404:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f800/0xdf5a7ac59f177254 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17724d expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.325692:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f800/0xdf5a7ac59f177254 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f17724d expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.325702:0:6404:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f800/0xdf5a7ac59f177254 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f17724d expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.325728:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f800/0xdf5a7ac59f177254 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17724d expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.325740:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6404:1876702074739840:0@lo:105: 00000100:00100000:1.0:1789762933.325776:0:6404:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762933.325816:0:6404:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37eebc60c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762933.325836:0:6402:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074739840 00000100:00100000:3.0:1789762933.325845:0:6402:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074739840 00000100:00100000:3.0:1789762933.325953:0:6402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57 00000100:00100000:3.0:1789762933.325967:0:6402:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6404:x1876702074739840:12345-0@lo:105: 00010000:00010000:3.0:1789762933.325982:0:6402:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.325996:0:6402:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.326008:0:6402:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.326017:0:6402:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.326026:0:6402:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.326036:0:6402:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.326063:0:6402:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c37d8cae800) 00010000:00010000:2.0:1789762933.326091:0:11692:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.326105:0:11692:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762933.326120:0:6402:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6404:x1876702074739840:12345-0@lo:105: Request processed in 155us (341us total) trans 0 rc 0/0 00000004:00080000:2.0:1789762933.326130:0:11692:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1789762933.326132:0:6402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57 00000100:00100000:1.0:1789762933.326157:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6404:1876702074739840:0@lo:105: 00010000:00010000:2.0:1789762933.326177:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.326178:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762933.326198:0:11692:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.326214:0:11692:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.326229:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067518720:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 621us (760us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762933.326249:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 88 00000100:00100000:2.0:1789762933.326250:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074739968:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1789762933.326336:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762933.326368:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff66e300 going to sleep for 11 seconds 00000100:00100000:3.0:1789762933.326376:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074739968 00000100:00100000:3.0:1789762933.326401:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074739968 00000100:00100000:3.0:1789762933.326467:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:3.0:1789762933.326549:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:11692:x1876702074739968:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1789762933.326584:0:6423:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762933.326604:0:6423:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f177262 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f17725b expref: -99 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.326634:0:6423:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f177262 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f17725b expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.326656:0:6423:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f177262 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17725b expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.326671:0:6423:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba800) 00010000:00010000:3.0:1789762933.326676:0:6423:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762933.326682:0:6423:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca315400, rc 0) 00000100:00100000:3.0:1789762933.326742:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074739968:12345-0@lo:101:mdt00_004.0 Request processed in 263us (406us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762933.326774:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:2.0:1789762933.326808:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074739968:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1789762933.326845:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177262 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.326881:0:11692:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177262 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762933.326905:0:11692:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1789762933.326990:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074740096:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789762933.327041:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762933.327071:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff66e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762933.327092:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074740096 00000100:00100000:1.0:1789762933.327101:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074740096 00000100:00100000:1.0:1789762933.327149:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:1.0:1789762933.327161:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074740096:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762933.327240:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074740096:12345-0@lo:1000:mdt00_004.0 Request processed in 82us (201us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762933.327249:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1789762933.327292:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074740096:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789762933.327346:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074740224:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789762933.327404:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762933.327436:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff66e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762933.327454:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074740224 00000100:00100000:1.0:1789762933.327463:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074740224 00000100:00100000:1.0:1789762933.327516:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:1.0:1789762933.327529:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074740224:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762933.327775:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074740224:12345-0@lo:1000:mdt00_004.0 Request processed in 249us (368us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789762933.327786:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00100000:2.0:1789762933.327824:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074740224:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789762933.328029:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074740352:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789762933.328074:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762933.328124:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff66e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762933.328125:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074740352 00000100:00100000:1.0:1789762933.328134:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074740352 00000100:00100000:1.0:1789762933.328176:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00100000:1.0:1789762933.328206:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ead180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074740352:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762933.328441:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ead180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074740352:12345-0@lo:1000:mdt00_004.0 Request processed in 236us (364us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789762933.328455:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00000100:00100000:2.0:1789762933.328500:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074740352:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789762933.328598:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eadc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074740480:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789762933.328645:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762933.328686:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff66ecc0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762933.328768:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074740480 00000100:00100000:3.0:1789762933.328779:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074740480 00000100:00100000:3.0:1789762933.328874:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:3.0:1789762933.328888:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eaed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074740480:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1789762933.329024:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eaed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074740480:12345-0@lo:1000:mdt00_004.0 Request processed in 137us (378us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762933.329037:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:2.0:1789762933.329055:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eadc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074740480:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789762933.329126:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eadc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074740608:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789762933.329170:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762933.329213:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff66ecc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762933.329227:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074740608 00000100:00100000:1.0:1789762933.329235:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074740608 00000100:00100000:1.0:1789762933.329280:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:1.0:1789762933.329295:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d69f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074740608:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762933.329370:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d69f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074740608:12345-0@lo:1000:mdt00_004.0 Request processed in 81us (204us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762933.329380:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:2.0:1789762933.329440:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eadc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074740608:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789762933.329521:0:11692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eadc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074740736:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789762933.329590:0:11692:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762933.329638:0:11692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ff66ecc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762933.329645:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074740736 00000100:00100000:1.0:1789762933.329654:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074740736 00000100:00100000:1.0:1789762933.329699:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:1.0:1789762933.329712:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8c8c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074740736:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789762933.329820:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8c8c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:11692:x1876702074740736:12345-0@lo:1000:mdt00_004.0 Request processed in 112us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762933.329828:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:2.0:1789762933.329886:0:11692:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eadc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:11692:1876702074740736:0@lo:1000:mdt00_004.0 00000040:00080000:2.0:1789762933.332587:0:11692:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].13, 984 off42200 00000040:00080000:2.0:1789762933.332790:0:11692:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].13, 1136 off43744 00000004:00080000:2.0:1789762933.332801:0:11692:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37ff66e9c0 with version 28 00000004:00080000:3.0:1789762933.332910:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 28 rpc_version 28 00000020:00100000:3.0:1789762933.332973:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1789762933.332984:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37f7f59180 x1876702074740992/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2416/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1789762933.333006:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f59180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074740992:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1789762933.333051:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762933.333085:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c5654d80 going to sleep for 11 seconds 00000100:00100000:2.0:1789762933.333106:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074740992 00000100:00100000:2.0:1789762933.333114:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074740992 00000100:00100000:2.0:1789762933.333166:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1789762933.333207:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7224:x1876702074740992:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762933.334605:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7224:x1876702074740992:12345-0@lo:1000:osp_up0-1.0 Request processed in 1421us (1545us total) trans 4294967445 rc 0/0 00000100:00100000:2.0:1789762933.334619:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 415 00000100:00100000:3.0:1789762933.334724:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f59180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074740992:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1789762933.334780:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177269 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.334814:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177269 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.334829:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177269 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762933.334844:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000704869d reply state 00000000766dd033 transno 4294967450 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177269 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.334877:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177269 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.334886:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762933.334891:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762933.334894:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762933.334899:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17:0x0]: returning size 16384 00000004:00080000:2.0:1789762933.334907:0:11692:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1789762933.334954:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.334968:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.334978:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.335007:0:11692:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1789762933.335012:0:11692:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1789762933.335018:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177262 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.335028:0:11692:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177262 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.335040:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762933.335053:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.335069:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae800 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:1.0:1789762933.335078:0:6408:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762933.335090:0:6408:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762933.335109:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e73480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15685:x1876702067518592:12345-192.168.203.37@tcp:36:mkdir.0 Request processed in 13203us (13410us total) trans 4294967450 rc 0/0 00000100:00100000:2.0:1789762933.335121:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 340 00010000:00010000:1.0:1789762933.335131:0:6408:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.335142:0:6408:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789762933.335202:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.335214:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae800/0xdf5a7ac59f17724d lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f177254 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762933.335225:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074741120:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762933.335238:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762933.335245:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762933.335250:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762933.335300:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762933.335391:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074741120 00000100:00100000:1.0:1789762933.335398:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074741120 00000100:00100000:1.0:1789762933.335455:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 69 00000100:00100000:1.0:1789762933.335467:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eadc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3427:x1876702074741120:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762933.335479:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762933.335498:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f800/0xdf5a7ac59f177254 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f17724d expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.335518:0:6406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762933.335522:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762933.335601:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eadc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+8:3427:x1876702074741120:12345-0@lo:103:ldlm_bl.0 Request processed in 136us (304us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762933.335632:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 69 00000100:00100000:2.0:1789762933.335677:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074741120:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789762933.344838:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762933.344975:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067518848 00000100:00100000:1.0:1789762933.344987:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067518848 00000100:00100000:1.0:1789762933.345055:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 341 00000100:00100000:1.0:1789762933.345073:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e81180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:15686:x1876702067518848:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:1.0:1789762933.345088:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762933.345103:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f800/0xdf5a7ac59f177270 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca85 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.345157:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177277 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.345171:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177277 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.345181:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177277 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762933.345208:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762933.345213:0:7744:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:1.0:1789762933.345232:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177277 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.345255:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f800/0xdf5a7ac59f177270 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca85 expref: 7 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.345273:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177277 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca85 expref: 6 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.345282:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762933.345285:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762933.345289:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d998f400, rc 0) 00000100:00100000:1.0:1789762933.345353:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e81180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15686:x1876702067518848:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 285us (519us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762933.345361:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 341 00000100:00100000:2.0:1789762933.347099:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762933.347153:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067519104 00000100:00100000:1.0:1789762933.347163:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067519104 00000100:00100000:1.0:1789762933.347213:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 342 00000100:00100000:1.0:1789762933.347225:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb450380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15686:x1876702067519104:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:1.0:1789762933.347238:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762933.347241:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1789762933.347257:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998f400/0xdf5a7ac59f177277 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca85 expref: 6 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.347276:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762933.347280:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762933.347290:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fa00/0xdf5a7ac59f17727e lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca8c expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.347380:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998ec00/0xdf5a7ac59f177285 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.347393:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998ec00/0xdf5a7ac59f177285 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.347402:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998ec00/0xdf5a7ac59f177285 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.347412:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.347422:0:6419:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f177262 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17725b expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.347432:0:6419:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.347457:0:6419:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f177262 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xdf5a7ac59f17725b expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.347471:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962e680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6419:1876702074741248:0@lo:104: 00000100:00100000:1.0:1789762933.347509:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762933.347572:0:6449:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074741248 00000100:00100000:1.0:1789762933.347573:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37eebc60c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762933.347586:0:6449:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074741248 00000100:00100000:3.0:1789762933.347641:0:6449:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 58 00000100:00100000:3.0:1789762933.347654:0:6449:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6419:x1876702074741248:12345-0@lo:104: 00010000:00010000:3.0:1789762933.347668:0:6449:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f177262 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.347750:0:6449:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1789762933.347761:0:6449:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6419:x1876702074741248:12345-0@lo:104: Request processed in 108us (249us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762933.347775:0:6449:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 58 00000100:00100000:1.0:1789762933.347789:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962e680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6419:1876702074741248:0@lo:104: 00010000:00010000:2.0:1789762933.347789:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762933.347810:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f177262 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.347811:0:6419:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.347843:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf800 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:2.0:1789762933.347851:0:7217:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xdf5a7ac59f177262 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762933.347887:0:7217:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xdf5a7ac59f177262 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.347956:0:7217:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f177262 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.347965:0:7217:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762933.348001:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f177262 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.348010:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf800/0xdf5a7ac59f17725b lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f177262 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.348020:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb452a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074741376:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762933.348034:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762933.348040:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762933.348043:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762933.348076:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762933.348133:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074741376 00000100:00100000:2.0:1789762933.348140:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074741376 00010000:00010000:2.0:1789762933.348164:0:7218:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f177262 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xdf5a7ac59f17725b expref: 8 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762933.348206:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 89 00000100:00100000:2.0:1789762933.348217:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c93050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074741376:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762933.348229:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:0.0:1789762933.362226:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967445 is committed 00010000:00010000:2.0:1789762933.364963:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315400/0xdf5a7ac59f177262 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xdf5a7ac59f17725b expref: 7 pid: 6423 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.364990:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba800) 00010000:00010000:2.0:1789762933.364996:0:7218:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.365006:0:7218:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.365013:0:7218:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.365047:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762933.365071:0:6419:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365084:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762933.365107:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c93050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074741376:12345-0@lo:103:ldlm_bl.0 Request processed in 16891us (17032us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762933.365119:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 89 00000100:00100000:3.0:1789762933.365154:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb452a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074741376:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789762933.365307:0:6419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9e70380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6419:1876702074741504:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789762933.365358:0:6419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762933.365400:0:6419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176fb40 going to sleep for 11 seconds 00000100:00100000:3.0:1789762933.365401:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074741504 00000100:00100000:3.0:1789762933.365410:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074741504 00000100:00100000:3.0:1789762933.365455:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 416 00000100:00100000:3.0:1789762933.365465:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e72a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6419:x1876702074741504:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762933.365566:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e72a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6419:x1876702074741504:12345-0@lo:1000:mdt00_001.0 Request processed in 98us (205us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762933.365578:0:6419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9e70380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6419:1876702074741504:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789762933.365582:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 416 00010000:00010000:0.0:1789762933.365633:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6000/0xdf5a7ac59f177293 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365652:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6000/0xdf5a7ac59f177293 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365661:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6000/0xdf5a7ac59f177293 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365674:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998ec00/0xdf5a7ac59f177285 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365682:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998ec00/0xdf5a7ac59f177285 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365690:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998ec00/0xdf5a7ac59f177285 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365696:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998ec00/0xdf5a7ac59f177285 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365702:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ec00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762933.365707:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998ec00/0xdf5a7ac59f177285 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365714:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998ec00/0xdf5a7ac59f177285 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365724:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762933.365727:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762933.365732:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998ec00/0xdf5a7ac59f177285 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365738:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998ec00/0xdf5a7ac59f177285 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365751:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365757:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365764:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365769:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998fe00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762933.365772:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365778:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365786:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72ba800) 00010000:00010000:0.0:1789762933.365790:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762933.365794:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365804:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fe00/0xdf5a7ac59f17728c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365871:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d998fa00/0xdf5a7ac59f17727e lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca8c expref: 7 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365895:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6000/0xdf5a7ac59f177293 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca8c expref: 6 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.365909:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37e37a7800) 00010000:00010000:0.0:1789762933.365912:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762933.365943:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca3a6000, rc 0) 00000100:00100000:0.0:1789762933.365995:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb450380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15686:x1876702067519104:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 18769us (18897us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762933.366003:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 342 00000100:00100000:3.0:1789762933.367625:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762933.367695:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067519360 00000100:00100000:0.0:1789762933.367702:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067519360 00000100:00100000:0.0:1789762933.367747:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 343 00000100:00100000:0.0:1789762933.367757:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e83100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15686:x1876702067519360:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762933.367772:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762933.367787:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca3a7e00/0xdf5a7ac59f17729a lrc: 2/0,0 mode: --/CW res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca93 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762933.367880:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17:0x0]: returning size 16384 00010000:00010000:0.0:1789762933.367905:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca3a7e00/0xdf5a7ac59f17729a lrc: 1/0,0 mode: --/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca93 expref: 8 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.367953:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:0.0:1789762933.367957:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762933.367962:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca3a7e00/0xdf5a7ac59f17729a lrc: 0/0,0 mode: --/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca93 expref: 8 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.367975:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca3a7e00, rc 0) 00000100:00100000:0.0:1789762933.368019:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e83100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15686:x1876702067519360:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 262us (396us total) trans 0 rc 301/301 00000100:00100000:0.0:1789762933.368034:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 343 00000100:00100000:1.0:1789762933.369367:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762933.369441:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067519488 00000100:00100000:0.0:1789762933.369448:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067519488 00000100:00100000:0.0:1789762933.369495:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 344 00000100:00100000:0.0:1789762933.369507:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:15686:x1876702067519488:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762933.369518:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762933.369531:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a7800/0xdf5a7ac59f1772a1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9ca9a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.369574:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6600/0xdf5a7ac59f1772a8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.369585:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6600/0xdf5a7ac59f1772a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.369594:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6600/0xdf5a7ac59f1772a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762933.369616:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762933.369620:0:13389:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762933.369637:0:13389:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6600/0xdf5a7ac59f1772a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.369658:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a7800/0xdf5a7ac59f1772a1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca9a expref: 8 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.369673:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6600/0xdf5a7ac59f1772a8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca9a expref: 7 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.369687:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762933.369690:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762933.369693:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca3a6600, rc 0) 00000100:00100000:0.0:1789762933.369735:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15686:x1876702067519488:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 231us (371us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762933.369745:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 344 00000100:00100000:2.0:1789762933.371099:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762933.371146:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067519744 00000100:00100000:2.0:1789762933.371152:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067519744 00000100:00100000:2.0:1789762933.371215:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 345 00000100:00100000:2.0:1789762933.371234:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb450e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15686:x1876702067519744:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762933.371251:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762933.371268:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f1772af lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9caa1 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371323:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1772b6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371341:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1772b6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371365:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1772b6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371392:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f1772bd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371406:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f1772bd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371415:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f1772bd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371696:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1772c4 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371714:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1772c4 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371725:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1772c4 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371740:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1772b6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371749:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1772b6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371757:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1772b6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371766:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1772b6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371778:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafc00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762933.371784:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1772b6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371793:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1772b6 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371805:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762933.371810:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762933.371815:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1772b6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371822:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1772b6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371841:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f1772bd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371849:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f1772bd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371881:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f1772bd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371908:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762933.371914:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f1772bd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371925:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f1772bd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371983:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37d3beeb00) 00010000:00010000:2.0:1789762933.371987:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762933.371991:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f1772bd lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.371997:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f1772bd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.372040:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f1772af lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caa1 expref: 9 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.372056:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1772c4 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caa1 expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.372064:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37e37a7800) 00010000:00010000:2.0:1789762933.372067:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762933.372070:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8cae000, rc 0) 00000100:00100000:2.0:1789762933.372125:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb450e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15686:x1876702067519744:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 897us (1033us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762933.372134:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 345 00000100:00100000:0.0:1789762933.373597:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762933.373724:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762933.373960:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067520128 00000100:00100000:2.0:1789762933.373967:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067520128 00000100:00100000:2.0:1789762933.374029:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 70 00000100:00100000:2.0:1789762933.374042:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e70e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067520128:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762933.374052:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762933.374066:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1772c4 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caa1 expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.374087:0:9878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37e37a7800) 00010000:00010000:2.0:1789762933.374092:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762933.374127:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e70e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067520128:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 91us (404us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762933.374135:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 70 00000100:00100000:2.0:1789762933.374172:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067520000 00000100:00100000:2.0:1789762933.374178:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067520000 00000100:00100000:2.0:1789762933.374228:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 346 00000100:00100000:2.0:1789762933.374243:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e70380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15686:x1876702067520000:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762933.374255:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762933.374268:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf000/0xdf5a7ac59f1772cb lrc: 2/0,0 mode: --/CR res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9caa8 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762933.374348:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17:0x0]: returning size 16384 00010000:00010000:2.0:1789762933.374508:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf000/0xdf5a7ac59f1772cb lrc: 1/0,0 mode: --/CR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caa8 expref: 9 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.374525:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762933.374529:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762933.374533:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf000/0xdf5a7ac59f1772cb lrc: 0/0,0 mode: --/CR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caa8 expref: 9 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.374569:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8caf000, rc 0) 00000100:00100000:2.0:1789762933.374602:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e70380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15686:x1876702067520000:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 364us (1010us total) trans 4294967451 rc 301/301 00000100:00100000:2.0:1789762933.374616:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 346 00000100:00100000:0.0:1789762933.376106:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762933.376137:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067520384 00000100:00100000:3.0:1789762933.376145:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067520384 00000100:00100000:3.0:1789762933.376187:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 347 00000100:00100000:3.0:1789762933.376198:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e71500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15686:x1876702067520384:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:3.0:1789762933.376248:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315000/0xdf5a7ac59f1772d9 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.376261:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315000/0xdf5a7ac59f1772d9 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.376271:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315000/0xdf5a7ac59f1772d9 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762933.376443:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000027621451 reply state 000000005dd51971 transno 4294967452 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315000/0xdf5a7ac59f1772d9 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.376461:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315000/0xdf5a7ac59f1772d9 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.376477:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:3.0:1789762933.376481:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:3.0:1789762933.376503:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762933.376510:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17:0x0]: returning size 16384 00000100:00100000:3.0:1789762933.376588:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e71500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:15686:x1876702067520384:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 388us (481us total) trans 4294967452 rc 0/0 00000100:00100000:3.0:1789762933.376603:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 347 00010000:00010000:0.0:1789762933.376681:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762933.377561:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762933.377602:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067520640 00000100:00100000:3.0:1789762933.377611:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067520640 00000100:00100000:3.0:1789762933.377666:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 101 00000100:00100000:3.0:1789762933.377681:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e71180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:15686:x1876702067520640:12345-192.168.203.37@tcp:35:lfs.0 00000100:00100000:3.0:1789762933.377847:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e71180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15686:x1876702067520640:12345-192.168.203.37@tcp:35:lfs.0 Request processed in 167us (284us total) trans 4294967453 rc 0/0 00000100:00100000:3.0:1789762933.377874:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 101 00000100:00100000:0.0:1789762933.383653:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762933.383751:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067520768 00000100:00100000:2.0:1789762933.383761:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067520768 00000100:00100000:2.0:1789762933.383829:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 348 00000100:00100000:2.0:1789762933.383850:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e71c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15687:x1876702067520768:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:2.0:1789762933.383891:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762933.383916:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f1772e0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9caaf expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.384034:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafa00/0xdf5a7ac59f1772e7 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.384054:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafa00/0xdf5a7ac59f1772e7 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.384068:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafa00/0xdf5a7ac59f1772e7 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762933.384098:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17:0x0]: returning size 16384 00000004:00200000:2.0:1789762933.384102:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762933.384119:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafa00/0xdf5a7ac59f1772e7 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.384148:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f1772e0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caaf expref: 13 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.384175:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafa00/0xdf5a7ac59f1772e7 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caaf expref: 12 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.384191:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762933.384196:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762933.384199:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8cafa00, rc 0) 00000100:00100000:2.0:1789762933.384262:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e71c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:15687:x1876702067520768:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 419us (613us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762933.384273:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 348 00000100:00100000:0.0:1789762933.385520:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762933.385607:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067521024 00000100:00100000:2.0:1789762933.385615:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067521024 00000100:00100000:2.0:1789762933.385664:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 349 00000100:00100000:2.0:1789762933.385678:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:15687:x1876702067521024:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:2.0:1789762933.385708:0:11692:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762933.385725:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafa00/0xdf5a7ac59f1772e7 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caaf expref: 12 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.385744:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762933.385749:0:11692:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762933.385763:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1772ee lrc: 3/0,1 mode: --/PW res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.385773:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1772ee lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.385782:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1772ee lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762933.386037:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a17806e6 reply state 00000000d7e6df8f transno 4294967454 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1772ee lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.386057:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1772ee lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.386065:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762933.386069:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:2.0:1789762933.386072:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762933.386078:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17:0x0]: returning size 16384 00000100:00100000:2.0:1789762933.386141:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:15687:x1876702067521024:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 466us (623us total) trans 4294967454 rc 0/0 00000100:00100000:2.0:1789762933.386150:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 349 00000100:00100000:0.0:1789762933.413432:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762933.413505:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067521152 00000100:00100000:3.0:1789762933.413514:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067521152 00000100:00100000:3.0:1789762933.413568:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 350 00000100:00100000:3.0:1789762933.413580:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15693:x1876702067521152:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762933.413596:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762933.413613:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315e00/0xdf5a7ac59f1772f5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cab6 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.413664:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f1772fc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.413679:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f1772fc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.413695:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f1772fc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.413710:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f177303 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.413718:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f177303 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.413725:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f177303 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414003:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f17730a lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414013:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f17730a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414020:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f17730a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762933.414038:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f17730a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414050:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f1772fc lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414057:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f1772fc lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414066:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f1772fc lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414083:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f1772fc lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414093:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762933.414099:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f1772fc lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414110:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f1772fc lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414124:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762933.414129:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762933.414135:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f1772fc lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414144:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314600/0xdf5a7ac59f1772fc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414154:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f177303 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414160:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f177303 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414166:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f177303 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414170:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762933.414173:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f177303 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414182:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f177303 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414189:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37c72bae00) 00010000:00010000:3.0:1789762933.414191:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762933.414194:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f177303 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414199:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca314c00/0xdf5a7ac59f177303 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414224:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315e00/0xdf5a7ac59f1772f5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cab6 expref: 9 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414235:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f17730a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cab6 expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.414241:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37e37a7800) 00010000:00010000:3.0:1789762933.414244:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762933.414246:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca315800, rc 0) 00000100:00100000:3.0:1789762933.414309:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:15693:x1876702067521152:12345-192.168.203.37@tcp:101:dd.0 Request processed in 732us (881us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762933.414318:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 350 00000100:00100000:0.0:1789762933.416097:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762933.416159:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067521280 00000100:00100000:2.0:1789762933.416167:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067521280 00000100:00100000:2.0:1789762933.416217:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 351 00000100:00100000:2.0:1789762933.416229:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:15693:x1876702067521280:12345-192.168.203.37@tcp:101:dd.0 00000100:00100000:0.0:1789762933.416242:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00010000:00010000:2.0:1789762933.416253:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762933.416269:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafa00/0xdf5a7ac59f177311 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cabd expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762933.416273:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067521408 00000100:00100000:3.0:1789762933.416282:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067521408 00010000:00010000:2.0:1789762933.416326:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177318 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.416338:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177318 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.416348:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177318 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762933.416371:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17:0x0]: returning size 16384 00000100:00100000:3.0:1789762933.416376:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 71 00000004:00200000:2.0:1789762933.416378:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000100:00100000:3.0:1789762933.416389:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067521408:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762933.416406:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762933.416409:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafa00/0xdf5a7ac59f177311 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cabd expref: 15 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.416427:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca315800/0xdf5a7ac59f17730a lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cab6 expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.416431:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177318 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cabd expref: 14 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.416454:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:3.0:1789762933.416457:0:8070:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37e37a7800) 00010000:00010000:2.0:1789762933.416458:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:2.0:1789762933.416465:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8caea00, rc 0) 00010000:00010000:3.0:1789762933.416466:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762933.416517:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067521408:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 126us (272us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762933.416518:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15693:x1876702067521280:12345-192.168.203.37@tcp:101:dd.0 Request processed in 287us (418us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762933.416539:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 71 00000100:00100000:2.0:1789762933.416540:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 351 00000100:00100000:0.0:1789762933.418157:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762933.418215:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067521664 00000100:00100000:2.0:1789762933.418225:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067521664 00000100:00100000:2.0:1789762933.418280:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 352 00000100:00100000:2.0:1789762933.418294:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15693:x1876702067521664:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762933.418308:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762933.418323:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf000/0xdf5a7ac59f17731f lrc: 2/0,0 mode: --/CW res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cac4 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.418404:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177326 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.418419:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177326 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.418429:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177326 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.418510:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f17732d lrc: 3/0,1 mode: --/PW res: [0x240000402:0x17:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.418520:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f17732d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.418529:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f17732d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762933.418674:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb451500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074741632:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1789762933.418729:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762933.418800:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37ca192480 going to sleep for 11 seconds 00000100:00100000:1.0:1789762933.418867:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074741632 00000100:00100000:1.0:1789762933.418877:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074741632 00000100:00100000:1.0:1789762933.418954:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:1.0:1789762933.418967:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb451880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074741632:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762933.419072:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb451880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074741632:12345-0@lo:1000:mdt00_002.0 Request processed in 108us (341us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762933.419082:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00100000:2.0:1789762933.419095:0:6420:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967445 gen 2 00000004:00080000:2.0:1789762933.419103:0:6420:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967445 committed transno 4294967445 00000100:00100000:2.0:1789762933.419129:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb451500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074741632:0@lo:1000:mdt00_002.0 00000004:00080000:2.0:1789762933.419493:0:6420:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0xe:0x0], index 0: 0 00010000:00010000:2.0:1789762933.419658:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f177334 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419672:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f177334 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419681:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f177334 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419713:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177326 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419721:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177326 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419730:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177326 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419737:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177326 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419743:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762933.419747:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177326 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419754:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177326 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419764:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762933.419767:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762933.419771:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177326 lrc: 3/0,0 mode: --/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419778:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177326 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762933.419790:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fe6383b7 reply state 00000000680ff8e0 transno 4294967455 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f17732d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419801:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f17732d lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419809:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x4009f790 (ffff9c37d3bee000) 00010000:00010000:2.0:1789762933.419811:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762933.419814:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762933.419839:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf000/0xdf5a7ac59f17731f lrc: 0/0,0 mode: --/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cac4 expref: 17 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419880:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f177334 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cac4 expref: 16 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.419890:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x18:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:2.0:1789762933.419893:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762933.419895:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8cae200, rc 0) 00000100:00100000:2.0:1789762933.419982:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:15693:x1876702067521664:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1688us (1825us total) trans 4294967455 rc 0/0 00000100:00100000:2.0:1789762933.419996:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 352 00000100:00100000:0.0:1789762933.425323:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762933.425447:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067521792 00000100:00100000:1.0:1789762933.425475:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067521792 00000100:00100000:1.0:1789762933.425558:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366381019739 time 366381124498 state 0x0 ref 1 00000100:00080000:0.0:1789762933.425566:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762933.425581:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 366431124498 time 366381124498 state 0x1 ref 1 00000100:00100000:1.0:1789762933.425619:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 2 deadline 366431124498 time 406672692969 state 0x0 ref 1 00000100:00100000:1.0:1789762933.425634:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067521792:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762933.425679:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789762933.425690:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177342 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.425709:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=406s v=(0 0) 00010000:00010000:1.0:1789762933.425716:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762933.425725:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177342 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.425751:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177342 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762933.456728:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968032 is committed 00002000:00010000:1.0:1789762933.456763:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:1.0:1789762933.456772:0:13301:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb mtime from disk: 0 -> 1789762933 00002000:00010000:1.0:1789762933.456777:0:13301:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb ctime from disk: 0 -> 1789762933 00002000:00010000:1.0:1789762933.456780:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:1.0:1789762933.456795:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177342 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.456812:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177342 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.456824:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177342 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.456833:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762933.456839:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177342 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.456850:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177342 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.456902:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177342 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.456953:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177342 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.457073:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067521792:12345-192.168.203.37@tcp:4:dd.0 Request processed in 31432us (31745us total) trans 4294968032 rc 0/0 00000100:00100000:1.0:1789762933.457093:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 924 00000100:00100000:1.0:1789762933.457110:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 2 deadline 366431124498 time 406672692969 state 0x2 ref 0 00000100:00100000:0.0:1789762933.461158:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762933.461226:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067521920 00000100:00100000:1.0:1789762933.461240:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067521920 00000100:00100000:1.0:1789762933.461297:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 2 deadline 366431124498 time 406672692969 state 0x0 ref 1 00000100:00100000:1.0:1789762933.461317:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 2 deadline 406722692969 time 406672692969 state 0x1 ref 1 00000100:00100000:1.0:1789762933.461356:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 1 deadline 406722692969 time 406708431217 state 0x0 ref 1 00000100:00100000:1.0:1789762933.461372:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067521920:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762933.461414:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 1048576, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x803 00010000:00010000:1.0:1789762933.461426:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177349 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.461445:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=406s v=(0 0) 00010000:00010000:1.0:1789762933.461451:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762933.461460:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177349 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.461476:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177349 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762933.483208:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968033 is committed 00002000:00010000:1.0:1789762933.483228:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:1.0:1789762933.483235:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:1.0:1789762933.483247:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177349 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.483259:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177349 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.483267:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177349 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.483273:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762933.483277:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177349 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.483284:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177349 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.483299:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177349 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.483305:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177349 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.483413:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067521920:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22040us (22257us total) trans 4294968033 rc 0/0 00000100:00100000:1.0:1789762933.483423:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 925 00000100:00100000:1.0:1789762933.483434:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 1 deadline 406722692969 time 406708431217 state 0x2 ref 0 00000100:00100000:0.0:1789762933.486269:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762933.486343:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067522048 00000100:00100000:1.0:1789762933.486357:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067522048 00000100:00100000:1.0:1789762933.486429:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 1 deadline 406722692969 time 406708431217 state 0x0 ref 1 00000100:00100000:1.0:1789762933.486447:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 1 deadline 406758431217 time 406708431217 state 0x1 ref 1 00000100:00100000:1.0:1789762933.486495:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406758431217 time 406733565882 state 0x0 ref 1 00000100:00100000:1.0:1789762933.486519:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067522048:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762933.486572:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 2097152, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x1007 00010000:00010000:1.0:1789762933.486583:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177350 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.486600:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=406s v=(0 0) 00010000:00010000:1.0:1789762933.486606:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762933.486613:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177350 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.486629:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177350 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762933.507326:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968034 is committed 00002000:00010000:1.0:1789762933.507378:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:1.0:1789762933.507388:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:1.0:1789762933.507405:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177350 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.507423:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177350 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.507436:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177350 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.507445:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762933.507451:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177350 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.507464:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177350 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.507488:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177350 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.507514:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177350 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.507640:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067522048:12345-192.168.203.37@tcp:4:dd.0 Request processed in 21122us (21374us total) trans 4294968034 rc 0/0 00000100:00100000:1.0:1789762933.507654:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 926 00000100:00100000:1.0:1789762933.507669:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406758431217 time 406733565882 state 0x2 ref 0 00000100:00100000:0.0:1789762933.512191:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762933.512296:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067522176 00000100:00100000:3.0:1789762933.512309:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067522176 00000100:00100000:3.0:1789762933.512373:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406758431217 time 406733565882 state 0x0 ref 1 00000100:00100000:3.0:1789762933.512392:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406783565882 time 406733565882 state 0x1 ref 1 00000100:00100000:1.0:1789762933.536640:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406783565882 time 406783707908 state 0x0 ref 1 00000100:00100000:1.0:1789762933.536663:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067522176:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762933.536709:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 3145728, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x1807 00010000:00010000:1.0:1789762933.536718:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177357 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.536735:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=406s v=(0 0) 00010000:00010000:1.0:1789762933.536740:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762933.536747:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177357 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.536763:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177357 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762933.559578:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968035 is committed 00002000:00010000:1.0:1789762933.559620:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:1.0:1789762933.559629:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:1.0:1789762933.559647:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177357 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.559671:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177357 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.559702:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177357 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.559719:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762933.559727:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177357 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.559745:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177357 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.559771:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177357 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.559781:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177357 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.559988:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067522176:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23252us (47721us total) trans 4294968035 rc 0/0 00000100:00100000:1.0:1789762933.560018:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 927 00000100:00100000:1.0:1789762933.560040:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406783565882 time 406783707908 state 0x2 ref 0 00000100:00100000:2.0:1789762933.564050:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762933.564115:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067522304 00000100:00100000:3.0:1789762933.564128:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067522304 00000100:00100000:3.0:1789762933.564188:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406783565882 time 406783707908 state 0x0 ref 1 00000100:00100000:3.0:1789762933.564206:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406833707908 time 406783707908 state 0x1 ref 1 00000100:00100000:2.0:1789762933.586790:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406833707908 time 406833853429 state 0x0 ref 1 00000100:00100000:2.0:1789762933.586817:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb451500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067522304:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762933.586896:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 4194304, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x2007 00010000:00010000:2.0:1789762933.586908:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17735e lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.586966:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=406s v=(0 0) 00010000:00010000:2.0:1789762933.586974:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762933.586986:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17735e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.587011:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17735e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762933.609228:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968036 is committed 00002000:00010000:1.0:1789762933.609292:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:1.0:1789762933.609302:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:1.0:1789762933.609317:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17735e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.609342:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17735e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.609354:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17735e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.609363:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762933.609369:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17735e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.609381:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17735e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.609403:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17735e lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.609429:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17735e lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.609530:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb451500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067522304:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22716us (45479us total) trans 4294968036 rc 0/0 00000100:00100000:1.0:1789762933.609543:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 928 00000100:00100000:1.0:1789762933.609559:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406833707908 time 406833853429 state 0x2 ref 0 00000100:00100000:0.0:1789762933.614127:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762933.614192:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067522432 00000100:00100000:1.0:1789762933.614207:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067522432 00000100:00100000:1.0:1789762933.614265:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406833707908 time 406833853429 state 0x0 ref 1 00000100:00100000:1.0:1789762933.614284:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406883853429 time 406833853429 state 0x1 ref 1 00000100:00100000:1.0:1789762933.636907:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406883853429 time 406883972820 state 0x0 ref 1 00000100:00100000:1.0:1789762933.636962:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067522432:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762933.637012:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 5242880, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x2807 00010000:00010000:1.0:1789762933.637023:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177365 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.637041:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=406s v=(0 0) 00010000:00010000:1.0:1789762933.637047:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762933.637055:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177365 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.637072:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177365 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762933.658772:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968037 is committed 00002000:00010000:2.0:1789762933.658907:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:2.0:1789762933.658948:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:2.0:1789762933.658966:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177365 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.658982:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177365 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.658995:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177365 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.659004:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762933.659010:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177365 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.659020:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177365 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.659043:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177365 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.659054:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f177365 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762933.659159:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067522432:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22200us (45034us total) trans 4294968037 rc 0/0 00000100:00100000:2.0:1789762933.659172:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 929 00000100:00100000:2.0:1789762933.659186:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406883853429 time 406883972820 state 0x2 ref 0 00000100:00100000:2.0:1789762933.663177:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762933.663282:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067522560 00000100:00100000:1.0:1789762933.663299:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067522560 00000100:00100000:1.0:1789762933.663378:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406883853429 time 406883972820 state 0x0 ref 1 00000100:00100000:1.0:1789762933.663400:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406933972820 time 406883972820 state 0x1 ref 1 00000100:00100000:1.0:1789762933.687016:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406933972820 time 406934076751 state 0x0 ref 1 00000100:00100000:1.0:1789762933.687049:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb451c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067522560:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762933.687136:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 6291456, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x3007 00010000:00010000:1.0:1789762933.687149:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17736c lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.687169:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=406s v=(0 0) 00010000:00010000:1.0:1789762933.687175:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762933.687183:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17736c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.687202:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17736c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762933.709740:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968038 is committed 00002000:00010000:2.0:1789762933.709836:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:2.0:1789762933.709846:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 12295 -> 14343 00010000:00010000:2.0:1789762933.709894:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17736c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.709912:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17736c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.709959:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17736c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.709970:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762933.709976:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17736c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.709988:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17736c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.710013:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17736c lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.710022:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17736c lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762933.710142:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb451c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067522560:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23101us (46964us total) trans 4294968038 rc 0/0 00000100:00100000:2.0:1789762933.710157:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 930 00000100:00100000:2.0:1789762933.710173:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406933972820 time 406934076751 state 0x2 ref 0 00000100:00100000:2.0:1789762933.714129:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762933.714235:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067522688 00000100:00100000:1.0:1789762933.714249:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067522688 00000100:00100000:1.0:1789762933.714326:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406933972820 time 406934076751 state 0x0 ref 1 00000100:00100000:1.0:1789762933.714355:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406984076751 time 406934076751 state 0x1 ref 1 00000100:00100000:3.0:1789762933.737120:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406984076751 time 406984183739 state 0x0 ref 1 00000100:00100000:3.0:1789762933.737147:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb452d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067522688:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762933.737200:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 7340032, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x3807 00010000:00010000:3.0:1789762933.737212:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f177373 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.737231:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=406s v=(0 0) 00010000:00010000:3.0:1789762933.737237:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762933.737246:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f177373 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.737262:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f177373 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762933.762109:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968039 is committed 00002000:00010000:2.0:1789762933.762261:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:2.0:1789762933.762272:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:2.0:1789762933.762290:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f177373 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.762308:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f177373 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.762321:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f177373 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.762330:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762933.762337:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f177373 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.762350:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f177373 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.762377:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f177373 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.762389:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f177373 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762933.762507:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb452d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067522688:12345-192.168.203.37@tcp:4:dd.0 Request processed in 25361us (48376us total) trans 4294968039 rc 0/0 00000100:00100000:2.0:1789762933.762524:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 931 00000100:00100000:2.0:1789762933.762541:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406984076751 time 406984183739 state 0x2 ref 0 00000100:00100000:2.0:1789762933.766716:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762933.766842:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067522816 00000100:00100000:1.0:1789762933.766885:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067522816 00000100:00100000:1.0:1789762933.766998:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 406984076751 time 406984183739 state 0x0 ref 1 00000100:00100000:1.0:1789762933.767029:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407034183739 time 406984183739 state 0x1 ref 1 00000100:00100000:3.0:1789762933.787291:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407034183739 time 407034354343 state 0x0 ref 1 00000100:00100000:3.0:1789762933.787323:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb450700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067522816:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762933.787383:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 8388608, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x4007 00010000:00010000:3.0:1789762933.787395:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f17737a lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.787415:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:3.0:1789762933.787422:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762933.787430:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f17737a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.787448:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f17737a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762933.809000:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968040 is committed 00002000:00010000:1.0:1789762933.809047:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 8388608 -> 9437184 00002000:00010000:1.0:1789762933.809061:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 16391 -> 18439 00010000:00010000:1.0:1789762933.809082:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f17737a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.809107:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f17737a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.809127:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f17737a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.809144:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762933.809153:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f17737a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.809171:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f17737a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.809196:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f17737a lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.809207:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f17737a lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.809341:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb450700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067522816:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22020us (42620us total) trans 4294968040 rc 0/0 00000100:00100000:1.0:1789762933.809362:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 932 00000100:00100000:1.0:1789762933.809378:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407034183739 time 407034354343 state 0x2 ref 0 00000100:00100000:0.0:1789762933.813402:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762933.813497:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067522944 00000100:00100000:2.0:1789762933.813511:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067522944 00000100:00100000:2.0:1789762933.813574:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407034183739 time 407034354343 state 0x0 ref 1 00000100:00100000:2.0:1789762933.813592:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407084354343 time 407034354343 state 0x1 ref 1 00000100:00100000:0.0:1789762933.837427:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407084354343 time 407084493237 state 0x0 ref 1 00000100:00100000:0.0:1789762933.837453:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067522944:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762933.837505:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 9437184, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x4807 00010000:00010000:0.0:1789762933.837516:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f177381 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.837534:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:0.0:1789762933.837540:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762933.837548:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f177381 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.837564:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f177381 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762933.860807:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 9437184 -> 10485760 00002000:00010000:2.0:1789762933.860816:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 18439 -> 20487 00010000:00010000:2.0:1789762933.860830:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f177381 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.860846:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f177381 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.860885:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f177381 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.860897:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a6400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762933.860902:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f177381 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.860913:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f177381 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.860981:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f177381 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.860992:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f177381 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762933.861063:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968041 is committed 00000100:00100000:2.0:1789762933.861097:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067522944:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23644us (47692us total) trans 4294968041 rc 0/0 00000100:00100000:2.0:1789762933.861113:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 933 00000100:00100000:2.0:1789762933.861127:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407084354343 time 407084493237 state 0x2 ref 0 00000100:00100000:2.0:1789762933.864952:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762933.865043:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067523072 00000100:00100000:0.0:1789762933.865056:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067523072 00000100:00100000:0.0:1789762933.865114:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407084354343 time 407084493237 state 0x0 ref 1 00000100:00100000:0.0:1789762933.865133:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407134493237 time 407084493237 state 0x1 ref 1 00000100:00100000:0.0:1789762933.887537:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407134493237 time 407134601095 state 0x0 ref 1 00000100:00100000:0.0:1789762933.887572:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067523072:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762933.887624:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 10485760, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x5007 00010000:00010000:0.0:1789762933.887635:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e9dd10/0xdf5a7ac59f177388 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.887654:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:0.0:1789762933.887660:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762933.887668:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e9dd10/0xdf5a7ac59f177388 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.887685:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1e9dd10/0xdf5a7ac59f177388 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762933.910640:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968042 is committed 00002000:00010000:1.0:1789762933.910712:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:1.0:1789762933.910722:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:1.0:1789762933.910740:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f1e9dd10/0xdf5a7ac59f177388 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.910756:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1e9dd10/0xdf5a7ac59f177388 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.910768:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1e9dd10/0xdf5a7ac59f177388 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.910778:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762933.910784:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e9dd10/0xdf5a7ac59f177388 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.910795:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1e9dd10/0xdf5a7ac59f177388 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.910818:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1e9dd10/0xdf5a7ac59f177388 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.910836:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1e9dd10/0xdf5a7ac59f177388 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.911048:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067523072:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23476us (46090us total) trans 4294968042 rc 0/0 00000100:00100000:1.0:1789762933.911070:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 934 00000100:00100000:1.0:1789762933.911087:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407134493237 time 407134601095 state 0x2 ref 0 00000100:00100000:2.0:1789762933.915118:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762933.915192:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067523200 00000100:00100000:1.0:1789762933.915206:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067523200 00000100:00100000:1.0:1789762933.915269:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407134493237 time 407134601095 state 0x0 ref 1 00000100:00100000:1.0:1789762933.915289:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407184601095 time 407134601095 state 0x1 ref 1 00000100:00100000:1.0:1789762933.937621:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407184601095 time 407184686439 state 0x0 ref 1 00000100:00100000:1.0:1789762933.937646:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb451f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067523200:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762933.937698:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 11534336, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x5807 00010000:00010000:1.0:1789762933.937709:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17738f lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.937728:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:1.0:1789762933.937734:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762933.937742:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17738f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.937759:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17738f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762933.959026:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968043 is committed 00002000:00010000:2.0:1789762933.959156:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 11534336 -> 12582912 00002000:00010000:2.0:1789762933.959165:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 22535 -> 24583 00010000:00010000:2.0:1789762933.959179:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17738f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.959204:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17738f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.959216:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17738f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.959225:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762933.959231:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17738f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.959242:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17738f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.959270:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17738f lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.959286:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f17738f lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762933.959403:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb451f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067523200:12345-192.168.203.37@tcp:4:dd.0 Request processed in 21759us (44283us total) trans 4294968043 rc 0/0 00000100:00100000:2.0:1789762933.959416:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 935 00000100:00100000:2.0:1789762933.959436:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407184601095 time 407184686439 state 0x2 ref 0 00000100:00100000:0.0:1789762933.963560:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762933.963629:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067523328 00000100:00100000:2.0:1789762933.963644:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067523328 00000100:00100000:2.0:1789762933.963702:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407184601095 time 407184686439 state 0x0 ref 1 00000100:00100000:2.0:1789762933.963720:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407234686439 time 407184686439 state 0x1 ref 1 00000100:00100000:1.0:1789762933.987760:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407234686439 time 407234825010 state 0x0 ref 1 00000100:00100000:1.0:1789762933.987787:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067523328:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762933.987871:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 12582912, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x6007 00010000:00010000:1.0:1789762933.987887:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f177396 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.987914:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:1.0:1789762933.987968:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762933.987977:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f177396 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.987999:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f177396 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762934.011233:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 12582912 -> 13631488 00000001:00080000:3.0:1789762934.011233:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968044 is committed 00002000:00010000:2.0:1789762934.011247:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 24583 -> 26631 00010000:00010000:2.0:1789762934.011267:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f177396 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.011283:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f177396 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.011294:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f177396 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.011303:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762934.011309:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f177396 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.011321:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f177396 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.011343:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f177396 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.011352:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c5411aa/0xdf5a7ac59f177396 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762934.011457:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067523328:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23673us (47896us total) trans 4294968044 rc 0/0 00000100:00100000:2.0:1789762934.011471:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 936 00000100:00100000:2.0:1789762934.011486:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407234686439 time 407234825010 state 0x2 ref 0 00000100:00100000:0.0:1789762934.015698:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762934.015785:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067523456 00000100:00100000:1.0:1789762934.015815:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067523456 00000100:00100000:1.0:1789762934.015926:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407234686439 time 407234825010 state 0x0 ref 1 00000100:00100000:1.0:1789762934.015999:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407284825010 time 407234825010 state 0x1 ref 1 00000100:00100000:2.0:1789762934.037989:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407284825010 time 407285049434 state 0x0 ref 1 00000100:00100000:2.0:1789762934.038022:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067523456:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762934.038074:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 13631488, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x6807 00010000:00010000:2.0:1789762934.038086:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17739d lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.038105:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:2.0:1789762934.038110:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762934.038118:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17739d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.038135:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17739d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762934.060328:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968045 is committed 00002000:00010000:1.0:1789762934.060407:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 13631488 -> 14680064 00002000:00010000:1.0:1789762934.060417:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:1.0:1789762934.060431:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17739d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.060449:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17739d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.060461:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17739d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.060471:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762934.060476:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17739d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.060488:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17739d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.060510:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17739d lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.060520:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17739d lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762934.060617:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067523456:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22601us (44918us total) trans 4294968045 rc 0/0 00000100:00100000:1.0:1789762934.060637:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 937 00000100:00100000:1.0:1789762934.060652:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407284825010 time 407285049434 state 0x2 ref 0 00000100:00100000:0.0:1789762934.064990:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762934.065064:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067523584 00000100:00100000:2.0:1789762934.065076:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067523584 00000100:00100000:2.0:1789762934.065131:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407284825010 time 407285049434 state 0x0 ref 1 00000100:00100000:2.0:1789762934.065150:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407335049434 time 407285049434 state 0x1 ref 1 00000100:00100000:0.0:1789762934.088156:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407335049434 time 407335216025 state 0x0 ref 1 00000100:00100000:0.0:1789762934.088190:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067523584:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762934.088254:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 14680064, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x7007 00010000:00010000:0.0:1789762934.088267:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f1773a4 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.088287:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:0.0:1789762934.088294:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762934.088302:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f1773a4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.088320:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f1773a4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762934.113640:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 14680064 -> 15728640 00000001:00080000:0.0:1789762934.113648:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968046 is committed 00002000:00010000:3.0:1789762934.113651:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:3.0:1789762934.113672:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f1773a4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.113691:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f1773a4 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.113703:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f1773a4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.113712:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a7c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762934.113717:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f1773a4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.113730:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f1773a4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.113753:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f1773a4 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.113779:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000de89688/0xdf5a7ac59f1773a4 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762934.114016:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067523584:12345-192.168.203.37@tcp:4:dd.0 Request processed in 25829us (49024us total) trans 4294968046 rc 0/0 00000100:00100000:3.0:1789762934.114040:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 938 00000100:00100000:3.0:1789762934.114058:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407335049434 time 407335216025 state 0x2 ref 0 00000100:00100000:0.0:1789762934.118600:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762934.118702:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067523712 00000100:00100000:2.0:1789762934.118716:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067523712 00000100:00100000:2.0:1789762934.118796:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407335049434 time 407335216025 state 0x0 ref 1 00000100:00100000:2.0:1789762934.118819:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407385216025 time 407335216025 state 0x1 ref 1 00000100:00100000:3.0:1789762934.138292:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407385216025 time 407385357586 state 0x0 ref 1 00000100:00100000:3.0:1789762934.138320:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067523712:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762934.138374:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 15728640, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x7807 00010000:00010000:3.0:1789762934.138386:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1773ab lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.138405:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:3.0:1789762934.138411:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762934.138419:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1773ab lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.138436:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1773ab lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762934.160793:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968047 is committed 00002000:00010000:1.0:1789762934.160906:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 15728640 -> 16777216 00002000:00010000:1.0:1789762934.160942:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 30727 -> 32775 00010000:00010000:1.0:1789762934.160960:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1773ab lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.160976:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1773ab lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.160988:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1773ab lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.160997:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762934.161003:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1773ab lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.161015:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1773ab lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.161037:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1773ab lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.161046:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1773ab lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762934.161154:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067523712:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22839us (42554us total) trans 4294968047 rc 0/0 00000100:00100000:1.0:1789762934.161169:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 939 00000100:00100000:1.0:1789762934.161183:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407385216025 time 407385357586 state 0x2 ref 0 00000100:00100000:0.0:1789762934.165183:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762934.165253:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067523840 00000100:00100000:1.0:1789762934.165270:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067523840 00000100:00100000:1.0:1789762934.165345:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407385216025 time 407385357586 state 0x0 ref 1 00000100:00100000:1.0:1789762934.165373:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407435357586 time 407385357586 state 0x1 ref 1 00000100:00100000:1.0:1789762934.188427:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407435357586 time 407435481540 state 0x0 ref 1 00000100:00100000:1.0:1789762934.188470:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067523840:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762934.188538:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 16777216, mtime: 0x6aad9d75, atime: 0x0, ctime: 0x6aad9d75, blocks: 0x8007 00010000:00010000:1.0:1789762934.188550:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1773b2 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.188571:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:1.0:1789762934.188577:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762934.188584:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1773b2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.188602:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1773b2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762934.211232:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968048 is committed 00002000:00010000:2.0:1789762934.211331:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 16777216 -> 17825792 00002000:00010000:2.0:1789762934.211340:0:8057:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb mtime from disk: 1789762933 -> 1789762934 00002000:00010000:2.0:1789762934.211344:0:8057:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb ctime from disk: 1789762933 -> 1789762934 00002000:00010000:2.0:1789762934.211347:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:2.0:1789762934.211362:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1773b2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.211378:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1773b2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.211391:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1773b2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.211400:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762934.211406:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1773b2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.211425:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1773b2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.211448:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1773b2 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.211457:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1773b2 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762934.211561:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067523840:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23104us (46378us total) trans 4294968048 rc 0/0 00000100:00100000:2.0:1789762934.211576:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 940 00000100:00100000:2.0:1789762934.211591:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407435357586 time 407435481540 state 0x2 ref 0 00000100:00100000:2.0:1789762934.215758:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762934.215841:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067523968 00000100:00100000:3.0:1789762934.215878:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067523968 00000100:00100000:3.0:1789762934.215987:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407435357586 time 407435481540 state 0x0 ref 1 00000100:00100000:3.0:1789762934.216013:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407485481540 time 407435481540 state 0x1 ref 1 00000100:00100000:1.0:1789762934.238633:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407485481540 time 407485693538 state 0x0 ref 1 00000100:00100000:1.0:1789762934.238664:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb452300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067523968:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762934.238715:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 17825792, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0x8807 00010000:00010000:1.0:1789762934.238726:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1773b9 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.238744:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:1.0:1789762934.238750:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762934.238758:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1773b9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.238774:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1773b9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762934.261017:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968049 is committed 00002000:00010000:2.0:1789762934.261039:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 17825792 -> 18874368 00002000:00010000:2.0:1789762934.261049:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 34823 -> 36871 00010000:00010000:2.0:1789762934.261064:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1773b9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.261081:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1773b9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.261093:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1773b9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.261111:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762934.261117:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1773b9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.261129:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1773b9 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.261151:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1773b9 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.261160:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f1773b9 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762934.261256:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb452300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067523968:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22597us (45499us total) trans 4294968049 rc 0/0 00000100:00100000:2.0:1789762934.261272:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 941 00000100:00100000:2.0:1789762934.261287:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407485481540 time 407485693538 state 0x2 ref 0 00000100:00100000:0.0:1789762934.265384:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762934.265480:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067524096 00000100:00100000:1.0:1789762934.265493:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067524096 00000100:00100000:1.0:1789762934.265566:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407485481540 time 407485693538 state 0x0 ref 1 00000100:00100000:1.0:1789762934.265593:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407535693538 time 407485693538 state 0x1 ref 1 00000100:00100000:1.0:1789762934.272178:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074741760:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762934.272259:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762934.272310:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074741760 00000100:00100000:0.0:1789762934.272322:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074741760 00000100:00100000:0.0:1789762934.272388:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 418 00000100:00100000:0.0:1789762934.272404:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074741760:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762934.272448:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:0.0:1789762934.272528:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074741760:12345-0@lo:41:osp-pre-0-1.0 Request processed in 129us (270us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762934.272538:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 418 00000100:00100000:1.0:1789762934.272635:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074741760:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762934.288707:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407535693538 time 407535771168 state 0x0 ref 1 00000100:00100000:1.0:1789762934.288735:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067524096:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762934.288785:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 18874368, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0x9007 00010000:00010000:1.0:1789762934.288797:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f1773c0 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.288816:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:1.0:1789762934.288822:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762934.288830:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f1773c0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.288873:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f1773c0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762934.310844:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968050 is committed 00002000:00010000:1.0:1789762934.311003:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:1.0:1789762934.311012:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:1.0:1789762934.311028:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f1773c0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.311045:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f1773c0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.311058:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f1773c0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.311076:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998fa00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762934.311082:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f1773c0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.311093:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f1773c0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.311115:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f1773c0 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.311124:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078194e37/0xdf5a7ac59f1773c0 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762934.311233:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067524096:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22503us (45849us total) trans 4294968050 rc 0/0 00000100:00100000:1.0:1789762934.311246:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 942 00000100:00100000:1.0:1789762934.311261:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407535693538 time 407535771168 state 0x2 ref 0 00000100:00100000:0.0:1789762934.315696:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762934.315817:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067524224 00000100:00100000:2.0:1789762934.315830:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067524224 00000100:00100000:2.0:1789762934.315958:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407535693538 time 407535771168 state 0x0 ref 1 00000100:00100000:2.0:1789762934.315979:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407585771168 time 407535771168 state 0x1 ref 1 00000100:00100000:2.0:1789762934.338819:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407585771168 time 407585872321 state 0x0 ref 1 00000100:00100000:2.0:1789762934.338867:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ec4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067524224:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762934.338948:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 19922944, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0x9807 00010000:00010000:2.0:1789762934.338959:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f1773c7 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.338977:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:2.0:1789762934.338983:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762934.338990:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f1773c7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.339006:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f1773c7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762934.360993:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968051 is committed 00002000:00010000:1.0:1789762934.361042:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 19922944 -> 20971520 00002000:00010000:1.0:1789762934.361052:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:1.0:1789762934.361069:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f1773c7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.361086:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f1773c7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.361107:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f1773c7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.361117:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762934.361123:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f1773c7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.361134:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f1773c7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.361157:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f1773c7 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.361166:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f1773c7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762934.361276:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ec4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067524224:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22433us (45579us total) trans 4294968051 rc 0/0 00000100:00100000:1.0:1789762934.361290:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 943 00000100:00100000:1.0:1789762934.361306:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407585771168 time 407585872321 state 0x2 ref 0 00000100:00100000:0.0:1789762934.365475:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762934.365579:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067524352 00000100:00100000:2.0:1789762934.365597:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067524352 00000100:00100000:2.0:1789762934.365676:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407585771168 time 407585872321 state 0x0 ref 1 00000100:00100000:2.0:1789762934.365700:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407635872321 time 407585872321 state 0x1 ref 1 00000100:00100000:2.0:1789762934.388961:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407635872321 time 407635996033 state 0x0 ref 1 00000100:00100000:2.0:1789762934.388993:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067524352:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762934.389051:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 20971520, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0xa007 00010000:00010000:2.0:1789762934.389063:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f1773ce lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.389085:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:2.0:1789762934.389091:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762934.389099:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f1773ce lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.389117:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f1773ce lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.400136:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762934.412040:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968052 is committed 00002000:00010000:1.0:1789762934.412092:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:1.0:1789762934.412101:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:1.0:1789762934.412117:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f1773ce lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.412135:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f1773ce lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.412147:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f1773ce lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.412157:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762934.412163:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f1773ce lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.412174:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f1773ce lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.412198:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f1773ce lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.412207:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f1773ce lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762934.412330:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067524352:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23339us (46853us total) trans 4294968052 rc 0/0 00000100:00100000:1.0:1789762934.412353:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 944 00000100:00100000:1.0:1789762934.412375:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407635872321 time 407635996033 state 0x2 ref 0 00000100:00100000:0.0:1789762934.416452:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762934.416537:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067524480 00000100:00100000:1.0:1789762934.416552:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067524480 00000100:00100000:1.0:1789762934.416631:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407635872321 time 407635996033 state 0x0 ref 1 00000100:00100000:1.0:1789762934.416651:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407685996033 time 407635996033 state 0x1 ref 1 00000100:00100000:3.0:1789762934.439071:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407685996033 time 407686137291 state 0x0 ref 1 00000100:00100000:3.0:1789762934.439098:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067524480:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762934.439147:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 22020096, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0xa807 00010000:00010000:3.0:1789762934.439157:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1773d5 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.439175:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:3.0:1789762934.439181:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762934.439188:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1773d5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.439205:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1773d5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762934.461621:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968053 is committed 00002000:00010000:1.0:1789762934.461744:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 22020096 -> 23068672 00002000:00010000:1.0:1789762934.461754:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:1.0:1789762934.461768:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1773d5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.461785:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1773d5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.461796:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1773d5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.461805:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762934.461811:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1773d5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.461823:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1773d5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.461872:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1773d5 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.461884:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1773d5 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762934.462043:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067524480:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22945us (45586us total) trans 4294968053 rc 0/0 00000100:00100000:1.0:1789762934.462063:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 945 00000100:00100000:1.0:1789762934.462085:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407685996033 time 407686137291 state 0x2 ref 0 00000100:00100000:0.0:1789762934.465760:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762934.465898:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067524608 00000100:00100000:1.0:1789762934.465911:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067524608 00000100:00100000:1.0:1789762934.465999:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407685996033 time 407686137291 state 0x0 ref 1 00000100:00100000:1.0:1789762934.466019:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407736137291 time 407686137291 state 0x1 ref 1 00000100:00100000:1.0:1789762934.489159:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407736137291 time 407736223695 state 0x0 ref 1 00000100:00100000:1.0:1789762934.489187:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067524608:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762934.489247:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 23068672, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0xb007 00010000:00010000:1.0:1789762934.489262:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f1773dc lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.489289:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:1.0:1789762934.489298:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762934.489310:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f1773dc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.489335:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f1773dc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762934.513567:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968054 is committed 00002000:00010000:2.0:1789762934.513662:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:2.0:1789762934.513673:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:2.0:1789762934.513698:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f1773dc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.513723:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f1773dc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.513736:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f1773dc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.513746:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762934.513753:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f1773dc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.513775:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f1773dc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.513800:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f1773dc lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.513809:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f1773dc lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762934.514037:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067524608:12345-192.168.203.37@tcp:4:dd.0 Request processed in 24845us (48267us total) trans 4294968054 rc 0/0 00000100:00100000:2.0:1789762934.514061:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 946 00000100:00100000:2.0:1789762934.514083:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407736137291 time 407736223695 state 0x2 ref 0 00000100:00100000:0.0:1789762934.518527:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762934.518620:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067524736 00000100:00100000:1.0:1789762934.518634:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067524736 00000100:00100000:1.0:1789762934.518702:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407736137291 time 407736223695 state 0x0 ref 1 00000100:00100000:1.0:1789762934.518722:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407786223695 time 407736223695 state 0x1 ref 1 00000100:00100000:2.0:1789762934.539287:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407786223695 time 407786347859 state 0x0 ref 1 00000100:00100000:2.0:1789762934.539322:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067524736:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762934.539406:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 24117248, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0xb807 00010000:00010000:2.0:1789762934.539418:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773e3 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.539437:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:2.0:1789762934.539443:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762934.539451:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773e3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.539469:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773e3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762934.561212:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968055 is committed 00002000:00010000:2.0:1789762934.561360:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 24117248 -> 25165824 00002000:00010000:2.0:1789762934.561371:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:2.0:1789762934.561387:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773e3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.561403:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773e3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.561415:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773e3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.561424:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762934.561430:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773e3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.561442:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773e3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.561464:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773e3 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.561473:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773e3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762934.561572:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067524736:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22258us (43045us total) trans 4294968055 rc 0/0 00000100:00100000:2.0:1789762934.561588:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 947 00000100:00100000:2.0:1789762934.561603:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407786223695 time 407786347859 state 0x2 ref 0 00000100:00100000:0.0:1789762934.565515:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762934.565651:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067524864 00000100:00100000:1.0:1789762934.565664:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067524864 00000100:00100000:1.0:1789762934.565725:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407786223695 time 407786347859 state 0x0 ref 1 00000100:00100000:1.0:1789762934.565744:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407836347859 time 407786347859 state 0x1 ref 1 00000100:00100000:0.0:1789762934.589456:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407836347859 time 407836513167 state 0x0 ref 1 00000100:00100000:0.0:1789762934.589500:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067524864:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762934.589573:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 25165824, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0xc007 00010000:00010000:0.0:1789762934.589585:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007942b887/0xdf5a7ac59f1773ea lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.589608:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:0.0:1789762934.589614:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762934.589623:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007942b887/0xdf5a7ac59f1773ea lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.589644:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007942b887/0xdf5a7ac59f1773ea lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762934.610572:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968056 is committed 00002000:00010000:3.0:1789762934.610672:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:3.0:1789762934.610684:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:3.0:1789762934.610705:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007942b887/0xdf5a7ac59f1773ea lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.610742:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007942b887/0xdf5a7ac59f1773ea lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.610757:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007942b887/0xdf5a7ac59f1773ea lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.610766:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a7400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762934.610772:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007942b887/0xdf5a7ac59f1773ea lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.610784:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007942b887/0xdf5a7ac59f1773ea lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.610807:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007942b887/0xdf5a7ac59f1773ea lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.610818:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007942b887/0xdf5a7ac59f1773ea lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762934.611001:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067524864:12345-192.168.203.37@tcp:4:dd.0 Request processed in 21510us (45486us total) trans 4294968056 rc 0/0 00000100:00100000:3.0:1789762934.611018:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 948 00000100:00100000:3.0:1789762934.611032:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407836347859 time 407836513167 state 0x2 ref 0 00000100:00100000:0.0:1789762934.614964:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762934.615045:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067524992 00000100:00100000:2.0:1789762934.615059:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067524992 00000100:00100000:2.0:1789762934.615118:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407836347859 time 407836513167 state 0x0 ref 1 00000100:00100000:2.0:1789762934.615137:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407886513167 time 407836513167 state 0x1 ref 1 00000100:00100000:2.0:1789762934.639557:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407886513167 time 407886622965 state 0x0 ref 1 00000100:00100000:2.0:1789762934.639582:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067524992:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762934.639630:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 26214400, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0xc807 00010000:00010000:2.0:1789762934.639640:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f1 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.639658:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:2.0:1789762934.639664:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762934.639671:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.639695:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762934.661978:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 26214400 -> 27262976 00002000:00010000:2.0:1789762934.661988:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:2.0:1789762934.662004:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.662021:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.662034:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762934.662040:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968057 is committed 00010000:00010000:2.0:1789762934.662044:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762934.662056:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.662076:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.662107:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f1 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.662124:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762934.662238:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067524992:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22656us (47313us total) trans 4294968057 rc 0/0 00000100:00100000:2.0:1789762934.662254:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 949 00000100:00100000:2.0:1789762934.662268:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407886513167 time 407886622965 state 0x2 ref 0 00000100:00100000:1.0:1789762934.666296:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762934.666414:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067525120 00000100:00100000:2.0:1789762934.666428:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067525120 00000100:00100000:2.0:1789762934.666483:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407886513167 time 407886622965 state 0x0 ref 1 00000100:00100000:2.0:1789762934.666501:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407936622965 time 407886622965 state 0x1 ref 1 00000100:00100000:2.0:1789762934.689665:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407936622965 time 407936722012 state 0x0 ref 1 00000100:00100000:2.0:1789762934.689704:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067525120:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762934.689774:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 27262976, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0xd007 00010000:00010000:2.0:1789762934.689787:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f8 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.689809:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:2.0:1789762934.689815:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762934.689823:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.689841:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762934.711420:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968058 is committed 00002000:00010000:2.0:1789762934.711516:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:2.0:1789762934.711527:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 53255 -> 55303 00010000:00010000:2.0:1789762934.711548:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.711572:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.711591:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.711608:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762934.711614:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.711627:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.711652:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f8 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.711663:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f1773f8 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762934.711767:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067525120:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22074us (45469us total) trans 4294968058 rc 0/0 00000100:00100000:2.0:1789762934.711784:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 950 00000100:00100000:2.0:1789762934.711800:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407936622965 time 407936722012 state 0x2 ref 0 00000100:00100000:0.0:1789762934.716063:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762934.716115:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067525248 00000100:00100000:3.0:1789762934.716130:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067525248 00000100:00100000:3.0:1789762934.716193:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407936622965 time 407936722012 state 0x0 ref 1 00000100:00100000:3.0:1789762934.716213:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407986722012 time 407936722012 state 0x1 ref 1 00000100:00100000:1.0:1789762934.739794:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407986722012 time 407986858710 state 0x0 ref 1 00000100:00100000:1.0:1789762934.739822:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067525248:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762934.739901:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 28311552, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0xd807 00010000:00010000:1.0:1789762934.739913:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1773ff lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.739965:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=407s v=(0 0) 00010000:00010000:1.0:1789762934.739972:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762934.739980:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1773ff lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.740005:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1773ff lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762934.761867:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968059 is committed 00002000:00010000:2.0:1789762934.762000:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 28311552 -> 29360128 00002000:00010000:2.0:1789762934.762010:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 55303 -> 57351 00010000:00010000:2.0:1789762934.762024:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1773ff lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.762040:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1773ff lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.762052:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1773ff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.762061:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762934.762067:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1773ff lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.762077:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1773ff lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.762100:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1773ff lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.762111:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f1773ff lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762934.762217:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067525248:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22399us (46157us total) trans 4294968059 rc 0/0 00000100:00100000:2.0:1789762934.762232:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 951 00000100:00100000:2.0:1789762934.762247:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407986722012 time 407986858710 state 0x2 ref 0 00000100:00100000:1.0:1789762934.766295:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762934.766410:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067525376 00000100:00100000:2.0:1789762934.766422:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067525376 00000100:00100000:2.0:1789762934.766478:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 407986722012 time 407986858710 state 0x0 ref 1 00000100:00100000:2.0:1789762934.766495:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408036858710 time 407986858710 state 0x1 ref 1 00000100:00100000:1.0:1789762934.789958:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408036858710 time 408036987610 state 0x0 ref 1 00000100:00100000:1.0:1789762934.789994:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067525376:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762934.790069:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 29360128, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0xe007 00010000:00010000:1.0:1789762934.790082:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177406 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.790112:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:1.0:1789762934.790119:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762934.790128:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177406 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.790149:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177406 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762934.806398:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967455 is committed 00000020:00080000:2.0:1789762934.806433:0:7221:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9c37ff66ee40 refcount 2 committed 0 result 0 batchid 13 00000020:00080000:2.0:1789762934.806441:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37f7b34400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9c37ff66e9c0 00000020:00080000:2.0:1789762934.806447:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x2:0x0].13 00000020:00080000:2.0:1789762934.806452:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37f7b34100 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9c37f79df400 00010000:00010000:0.0:1789762934.806457:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177269 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1789762934.806457:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x2:0x0].13 00010000:00010000:0.0:1789762934.806482:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177269 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806495:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177269 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806505:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762934.806513:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177269 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1789762934.806515:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 13 committed batchid 12 00010000:00010000:0.0:1789762934.806538:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177269 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806568:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:0.0:1789762934.806573:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762934.806579:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177269 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806588:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf400/0xdf5a7ac59f177269 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806620:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315000/0xdf5a7ac59f1772d9 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806630:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315000/0xdf5a7ac59f1772d9 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806639:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315000/0xdf5a7ac59f1772d9 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806648:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762934.806655:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315000/0xdf5a7ac59f1772d9 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806664:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315000/0xdf5a7ac59f1772d9 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806680:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:0.0:1789762934.806685:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789762934.806689:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315000/0xdf5a7ac59f1772d9 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806703:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca315000/0xdf5a7ac59f1772d9 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806723:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1772ee lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806731:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1772ee lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806739:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1772ee lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806747:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762934.806753:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1772ee lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806762:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1772ee lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806779:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:0.0:1789762934.806783:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762934.806787:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1772ee lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806795:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1772ee lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806819:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f17732d lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806828:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f17732d lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806838:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f17732d lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806874:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762934.806881:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f17732d lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806890:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f17732d lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806907:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x4009f790 (ffff9c37d3bee000) 00010000:00010000:0.0:1789762934.806911:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762934.806969:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f17732d lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.806980:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caf600/0xdf5a7ac59f17732d lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1789762934.807112:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 12 00000001:00080000:1.0:1789762934.815758:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968060 is committed 00002000:00010000:2.0:1789762934.815902:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 29360128 -> 30408704 00002000:00010000:2.0:1789762934.815911:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 57351 -> 59399 00010000:00010000:2.0:1789762934.815947:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177406 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.815970:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177406 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.816001:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177406 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.816017:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762934.816026:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177406 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.816045:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177406 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.816070:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177406 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.816081:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177406 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762934.816176:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067525376:12345-192.168.203.37@tcp:4:dd.0 Request processed in 26189us (49879us total) trans 4294968060 rc 0/0 00000100:00100000:2.0:1789762934.816188:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 952 00000100:00100000:2.0:1789762934.816203:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408036858710 time 408036987610 state 0x2 ref 0 00000100:00100000:2.0:1789762934.819784:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762934.819975:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067525504 00000100:00100000:1.0:1789762934.819989:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067525504 00000100:00100000:1.0:1789762934.820052:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408036858710 time 408036987610 state 0x0 ref 1 00000100:00100000:1.0:1789762934.820072:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408086987610 time 408036987610 state 0x1 ref 1 00000100:00100000:2.0:1789762934.840046:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408086987610 time 408087112545 state 0x0 ref 1 00000100:00100000:2.0:1789762934.840071:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb452680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067525504:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762934.840120:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 30408704, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0xe807 00010000:00010000:2.0:1789762934.840131:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f17740d lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.840149:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:2.0:1789762934.840155:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762934.840163:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f17740d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.840180:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f17740d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762934.861973:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968061 is committed 00002000:00010000:2.0:1789762934.862095:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 30408704 -> 31457280 00002000:00010000:2.0:1789762934.862105:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 59399 -> 61447 00010000:00010000:2.0:1789762934.862119:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f17740d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.862135:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f17740d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.862147:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f17740d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.862156:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caec00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762934.862162:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f17740d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.862181:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f17740d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.862202:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f17740d lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.862212:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f17740d lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762934.862308:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb452680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067525504:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22241us (42528us total) trans 4294968061 rc 0/0 00000100:00100000:2.0:1789762934.862322:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 953 00000100:00100000:2.0:1789762934.862337:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408086987610 time 408087112545 state 0x2 ref 0 00000100:00100000:0.0:1789762934.866035:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762934.866160:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067525632 00000100:00100000:2.0:1789762934.866173:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067525632 00000100:00100000:2.0:1789762934.866227:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408086987610 time 408087112545 state 0x0 ref 1 00000100:00100000:2.0:1789762934.866245:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408137112545 time 408087112545 state 0x1 ref 1 00000100:00100000:1.0:1789762934.890241:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408137112545 time 408137299102 state 0x0 ref 1 00000100:00100000:1.0:1789762934.890279:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067525632:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762934.890350:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 31457280, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0xf007 00010000:00010000:1.0:1789762934.890366:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f177414 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.890394:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:1.0:1789762934.890403:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762934.890415:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f177414 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.890443:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f177414 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762934.912579:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968062 is committed 00002000:00010000:3.0:1789762934.912643:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 31457280 -> 32505856 00002000:00010000:3.0:1789762934.912656:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 61447 -> 63495 00010000:00010000:3.0:1789762934.912678:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f177414 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.912705:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f177414 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.912722:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f177414 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.912733:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762934.912740:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f177414 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.912753:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f177414 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.912786:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f177414 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.912797:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062897897/0xdf5a7ac59f177414 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762934.912968:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067525632:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22699us (46931us total) trans 4294968062 rc 0/0 00000100:00100000:3.0:1789762934.912983:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 954 00000100:00100000:3.0:1789762934.913000:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408137112545 time 408137299102 state 0x2 ref 0 00000100:00100000:2.0:1789762934.917277:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762934.917344:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067525760 00000100:00100000:3.0:1789762934.917357:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067525760 00000100:00100000:3.0:1789762934.917417:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408137112545 time 408137299102 state 0x0 ref 1 00000100:00100000:3.0:1789762934.917436:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408187299102 time 408137299102 state 0x1 ref 1 00000100:00100000:3.0:1789762934.940311:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408187299102 time 408187375556 state 0x0 ref 1 00000100:00100000:3.0:1789762934.940337:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb450000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067525760:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762934.940390:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 32505856, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0xf807 00010000:00010000:3.0:1789762934.940401:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17741b lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.940443:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:3.0:1789762934.940449:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762934.940457:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17741b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762934.940474:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17741b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762934.962458:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968063 is committed 00002000:00010000:1.0:1789762934.962463:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 32505856 -> 33554432 00002000:00010000:1.0:1789762934.962474:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 63495 -> 65543 00010000:00010000:1.0:1789762934.962491:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17741b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.962508:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17741b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.962529:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17741b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.962538:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762934.962544:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17741b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.962555:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17741b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.962584:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17741b lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.962600:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17741b lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762934.962712:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb450000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067525760:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22378us (45437us total) trans 4294968063 rc 0/0 00000100:00100000:1.0:1789762934.962725:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 955 00000100:00100000:1.0:1789762934.962740:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408187299102 time 408187375556 state 0x2 ref 0 00000100:00100000:0.0:1789762934.967015:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762934.967068:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067525888 00000100:00100000:2.0:1789762934.967083:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067525888 00000100:00100000:2.0:1789762934.967148:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408187299102 time 408187375556 state 0x0 ref 1 00000100:00100000:2.0:1789762934.967173:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408237375556 time 408187375556 state 0x1 ref 1 00000100:00100000:1.0:1789762934.990489:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408237375556 time 408237553247 state 0x0 ref 1 00000100:00100000:1.0:1789762934.990517:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067525888:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762934.990578:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 33554432, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0x10007 00010000:00010000:1.0:1789762934.990590:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177422 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.990613:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:1.0:1789762934.990619:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762934.990627:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177422 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.990647:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177422 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762935.012532:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968064 is committed 00002000:00010000:2.0:1789762935.012643:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:2.0:1789762935.012653:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 65543 -> 67591 00010000:00010000:2.0:1789762935.012669:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177422 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.012685:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177422 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.012698:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177422 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.012707:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.012713:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177422 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.012726:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177422 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.012749:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177422 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.012759:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177422 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.012896:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067525888:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22382us (45884us total) trans 4294968064 rc 0/0 00000100:00100000:2.0:1789762935.012911:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 956 00000100:00100000:2.0:1789762935.012988:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408237375556 time 408237553247 state 0x2 ref 0 00000100:00100000:1.0:1789762935.016865:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762935.017012:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067526016 00000100:00100000:3.0:1789762935.017029:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067526016 00000100:00080000:0.0:1789762935.017111:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762935.017124:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408237375556 time 408237553247 state 0x0 ref 1 00000100:00100000:3.0:1789762935.017145:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408287553247 time 408237553247 state 0x1 ref 1 00000100:00100000:1.0:1789762935.040654:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408287553247 time 408287713762 state 0x0 ref 1 00000100:00100000:1.0:1789762935.040688:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067526016:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762935.040754:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 34603008, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0x10807 00010000:00010000:1.0:1789762935.040766:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177429 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.040787:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:1.0:1789762935.040793:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762935.040801:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177429 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.040826:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177429 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762935.063238:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968065 is committed 00002000:00010000:2.0:1789762935.063238:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:2.0:1789762935.063251:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 67591 -> 69639 00010000:00010000:2.0:1789762935.063272:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177429 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.063290:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177429 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.063304:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177429 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.063314:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.063320:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177429 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.063333:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177429 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.063355:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177429 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.063374:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003366fa5/0xdf5a7ac59f177429 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.063485:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067526016:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22801us (46640us total) trans 4294968065 rc 0/0 00000100:00100000:2.0:1789762935.063506:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 957 00000100:00100000:2.0:1789762935.063527:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408287553247 time 408287713762 state 0x2 ref 0 00000100:00100000:1.0:1789762935.067456:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762935.067541:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067526144 00000100:00100000:3.0:1789762935.067555:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067526144 00000100:00100000:3.0:1789762935.067616:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408287553247 time 408287713762 state 0x0 ref 1 00000100:00100000:3.0:1789762935.067635:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408337713762 time 408287713762 state 0x1 ref 1 00000100:00100000:3.0:1789762935.090771:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408337713762 time 408337827136 state 0x0 ref 1 00000100:00100000:3.0:1789762935.090816:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067526144:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762935.090972:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 35651584, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0x11007 00010000:00010000:3.0:1789762935.090990:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177430 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.091012:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:3.0:1789762935.091018:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762935.091027:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177430 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.091047:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177430 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762935.113725:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968066 is committed 00002000:00010000:2.0:1789762935.113874:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 35651584 -> 36700160 00002000:00010000:2.0:1789762935.113885:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 69639 -> 71687 00010000:00010000:2.0:1789762935.113910:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177430 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.113963:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177430 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.113976:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177430 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.113985:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.113992:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177430 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.114004:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177430 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.114037:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177430 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.114047:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177430 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.114179:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067526144:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23376us (46721us total) trans 4294968066 rc 0/0 00000100:00100000:2.0:1789762935.114195:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 958 00000100:00100000:2.0:1789762935.114212:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408337713762 time 408337827136 state 0x2 ref 0 00000100:00100000:0.0:1789762935.118343:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.118416:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067526272 00000100:00100000:2.0:1789762935.118430:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067526272 00000100:00100000:2.0:1789762935.118494:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408337713762 time 408337827136 state 0x0 ref 1 00000100:00100000:2.0:1789762935.118513:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408387827136 time 408337827136 state 0x1 ref 1 00000100:00100000:2.0:1789762935.140895:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408387827136 time 408387948313 state 0x0 ref 1 00000100:00100000:2.0:1789762935.140962:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067526272:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762935.141028:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 36700160, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0x11807 00010000:00010000:2.0:1789762935.141041:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f177437 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.141070:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:2.0:1789762935.141076:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762935.141083:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f177437 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.141101:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f177437 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762935.162760:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968067 is committed 00002000:00010000:2.0:1789762935.162888:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 36700160 -> 37748736 00002000:00010000:2.0:1789762935.162901:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 71687 -> 73735 00010000:00010000:2.0:1789762935.162960:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f177437 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.162980:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f177437 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.162992:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f177437 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.163001:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.163007:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f177437 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.163019:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f177437 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.163050:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f177437 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.163060:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f177437 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.163163:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067526272:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22212us (44823us total) trans 4294968067 rc 0/0 00000100:00100000:2.0:1789762935.163178:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 959 00000100:00100000:2.0:1789762935.163194:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408387827136 time 408387948313 state 0x2 ref 0 00000100:00100000:0.0:1789762935.167378:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.167477:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067526400 00000100:00100000:2.0:1789762935.167491:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067526400 00000100:00100000:2.0:1789762935.167551:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408387827136 time 408387948313 state 0x0 ref 1 00000100:00100000:2.0:1789762935.167570:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408437948313 time 408387948313 state 0x1 ref 1 00000100:00100000:2.0:1789762935.191011:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408437948313 time 408438074501 state 0x0 ref 1 00000100:00100000:2.0:1789762935.191040:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067526400:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762935.191095:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 37748736, mtime: 0x6aad9d76, atime: 0x0, ctime: 0x6aad9d76, blocks: 0x12007 00010000:00010000:2.0:1789762935.191106:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f17743e lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.191134:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:2.0:1789762935.191141:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762935.191148:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f17743e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.191166:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f17743e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762935.213503:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968068 is committed 00002000:00010000:2.0:1789762935.213551:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:2.0:1789762935.213565:0:14171:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb mtime from disk: 1789762934 -> 1789762935 00002000:00010000:2.0:1789762935.213572:0:14171:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb ctime from disk: 1789762934 -> 1789762935 00002000:00010000:2.0:1789762935.213578:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 73735 -> 75783 00010000:00010000:2.0:1789762935.213598:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f17743e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.213618:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f17743e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.213631:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f17743e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.213641:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.213647:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f17743e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.213660:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f17743e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.213686:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f17743e lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.213697:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005895e81e/0xdf5a7ac59f17743e lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.213816:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067526400:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22778us (46437us total) trans 4294968068 rc 0/0 00000100:00100000:2.0:1789762935.213862:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 960 00000100:00100000:2.0:1789762935.213887:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408437948313 time 408438074501 state 0x2 ref 0 00000100:00100000:1.0:1789762935.218529:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.218602:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067526528 00000100:00100000:2.0:1789762935.218616:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067526528 00000100:00100000:2.0:1789762935.218681:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408437948313 time 408438074501 state 0x0 ref 1 00000100:00100000:2.0:1789762935.218700:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408488074501 time 408438074501 state 0x1 ref 1 00000100:00100000:2.0:1789762935.240214:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.240287:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762935.240309:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067526784 00000100:00100000:0.0:1789762935.240322:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067526784 00000100:00100000:2.0:1789762935.240355:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762935.240402:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067526656 00000100:00100000:2.0:1789762935.240416:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762935.240420:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067526656 00000100:00100000:3.0:1789762935.240451:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067526912 00000100:00100000:0.0:1789762935.240458:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067527040 00000100:00100000:3.0:1789762935.240464:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067526912 00000100:00100000:0.0:1789762935.240485:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067527040 00000100:00100000:2.0:1789762935.240497:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762935.240526:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 385 00000100:00100000:1.0:1789762935.240553:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15c7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067526656:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762935.240555:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067527168 00000100:00100000:3.0:1789762935.240565:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067527168 00000100:00100000:0.0:1789762935.240571:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 618 00000100:00100000:2.0:1789762935.240594:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 619 00000100:00100000:0.0:1789762935.240596:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb453b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067526784:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762935.240625:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb453480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067526912:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762935.240642:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 620 00000100:00100000:1.0:1789762935.240654:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15c7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067526656:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 108us (365us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762935.240668:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 385 00000100:00100000:0.0:1789762935.240684:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb453b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067526784:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 89us (465us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762935.240684:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb451180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067527040:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762935.240716:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb453480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067526912:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 99us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762935.240718:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 618 00000100:00100000:2.0:1789762935.240740:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 619 00000100:00080000:1.0:1789762935.240758:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789762935.240770:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762935.240775:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 621 00000100:00100000:3.0:1789762935.240787:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb451180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067527040:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 119us (368us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762935.240802:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb450a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067527168:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762935.240825:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 620 00000100:00100000:0.0:1789762935.240911:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb450a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067527168:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 115us (419us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762935.240976:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 621 00000100:00100000:3.0:1789762935.241103:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408488074501 time 408488171695 state 0x0 ref 1 00000100:00100000:3.0:1789762935.241124:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067526528:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762935.241241:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 38797312, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x12807 00010000:00010000:3.0:1789762935.241255:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f177445 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.241275:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:3.0:1789762935.241281:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762935.241288:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f177445 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.241307:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f177445 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762935.263572:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968069 is committed 00002000:00010000:2.0:1789762935.263592:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 38797312 -> 39845888 00002000:00010000:2.0:1789762935.263602:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 75783 -> 77831 00010000:00010000:2.0:1789762935.263619:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f177445 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.263637:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f177445 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.263649:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f177445 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.263659:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.263665:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f177445 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.263676:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f177445 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.263698:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f177445 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.263708:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f177445 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.263823:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067526528:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22695us (45290us total) trans 4294968069 rc 0/0 00000100:00100000:2.0:1789762935.263863:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 961 00000100:00100000:2.0:1789762935.263886:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408488074501 time 408488171695 state 0x2 ref 0 00000100:00100000:1.0:1789762935.267541:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.267613:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067527296 00000100:00100000:2.0:1789762935.267625:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067527296 00000100:00100000:2.0:1789762935.267681:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408488074501 time 408488171695 state 0x0 ref 1 00000100:00100000:2.0:1789762935.267700:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408538171695 time 408488171695 state 0x1 ref 1 00000100:00100000:3.0:1789762935.291291:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408538171695 time 408538354713 state 0x0 ref 1 00000100:00100000:3.0:1789762935.291321:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067527296:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762935.291377:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 39845888, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x13007 00010000:00010000:3.0:1789762935.291388:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f17744c lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.291408:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:3.0:1789762935.291431:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762935.291440:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f17744c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.291457:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f17744c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762935.314758:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 39845888 -> 40894464 00000001:00080000:0.0:1789762935.314762:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968070 is committed 00002000:00010000:1.0:1789762935.314771:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 77831 -> 79879 00010000:00010000:1.0:1789762935.314793:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f17744c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.314819:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f17744c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.314860:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f17744c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.314880:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762935.314892:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f17744c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.314911:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f17744c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.314973:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f17744c lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.314984:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f17744c lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762935.315097:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067527296:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23781us (47555us total) trans 4294968070 rc 0/0 00000100:00100000:1.0:1789762935.315110:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 962 00000100:00100000:1.0:1789762935.315127:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408538171695 time 408538354713 state 0x2 ref 0 00000100:00100000:0.0:1789762935.319124:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.319232:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067527424 00000100:00100000:2.0:1789762935.319246:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067527424 00000100:00100000:2.0:1789762935.319326:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408538171695 time 408538354713 state 0x0 ref 1 00000100:00100000:2.0:1789762935.319346:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408588354713 time 408538354713 state 0x1 ref 1 00000100:00100000:1.0:1789762935.341434:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408588354713 time 408588500920 state 0x0 ref 1 00000100:00100000:1.0:1789762935.341457:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067527424:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762935.341513:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 40894464, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x13807 00010000:00010000:1.0:1789762935.341528:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177453 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.341554:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:1.0:1789762935.341562:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762935.341574:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177453 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.341598:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177453 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762935.363493:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968071 is committed 00002000:00010000:2.0:1789762935.363556:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 40894464 -> 41943040 00002000:00010000:2.0:1789762935.363568:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 79879 -> 81927 00010000:00010000:2.0:1789762935.363583:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177453 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.363599:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177453 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.363618:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177453 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.363628:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.363634:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177453 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.363644:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177453 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.363666:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177453 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.363676:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177453 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.363782:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067527424:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22326us (44658us total) trans 4294968071 rc 0/0 00000100:00100000:2.0:1789762935.363795:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 963 00000100:00100000:2.0:1789762935.363809:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408588354713 time 408588500920 state 0x2 ref 0 00000100:00100000:2.0:1789762935.367708:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762935.367796:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067527552 00000100:00100000:3.0:1789762935.367808:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067527552 00000100:00100000:3.0:1789762935.367890:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408588354713 time 408588500920 state 0x0 ref 1 00000100:00100000:3.0:1789762935.367908:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408638500920 time 408588500920 state 0x1 ref 1 00000100:00100000:1.0:1789762935.391595:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408638500920 time 408638655468 state 0x0 ref 1 00000100:00100000:1.0:1789762935.391627:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb453100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067527552:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762935.391689:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 41943040, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x14007 00010000:00010000:1.0:1789762935.391700:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17745a lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.391722:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:1.0:1789762935.391728:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762935.391736:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17745a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.391755:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17745a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762935.414805:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968072 is committed 00002000:00010000:2.0:1789762935.414863:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 41943040 -> 42991616 00002000:00010000:2.0:1789762935.414874:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 81927 -> 83975 00010000:00010000:2.0:1789762935.414893:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17745a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.414912:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17745a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.414956:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17745a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.414967:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.414973:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17745a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.414985:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17745a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.415009:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17745a lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.415018:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb714ec/0xdf5a7ac59f17745a lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.415127:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb453100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067527552:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23507us (47418us total) trans 4294968072 rc 0/0 00000100:00100000:2.0:1789762935.415142:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 964 00000100:00100000:2.0:1789762935.415158:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408638500920 time 408638655468 state 0x2 ref 0 00000100:00100000:0.0:1789762935.419340:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762935.419440:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067527680 00000100:00100000:1.0:1789762935.419456:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067527680 00000100:00100000:1.0:1789762935.419521:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408638500920 time 408638655468 state 0x0 ref 1 00000100:00100000:1.0:1789762935.419540:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408688655468 time 408638655468 state 0x1 ref 1 00010000:00010000:1.0:1789762935.424211:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762935.441768:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408688655468 time 408688834669 state 0x0 ref 1 00000100:00100000:3.0:1789762935.441795:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067527680:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762935.441876:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 42991616, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x14807 00010000:00010000:3.0:1789762935.441887:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f177461 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.441906:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:3.0:1789762935.441912:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762935.441950:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f177461 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.441969:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f177461 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762935.465067:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968073 is committed 00002000:00010000:2.0:1789762935.465137:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 42991616 -> 44040192 00002000:00010000:2.0:1789762935.465146:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 83975 -> 86023 00010000:00010000:2.0:1789762935.465161:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f177461 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.465177:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f177461 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.465189:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f177461 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.465198:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.465204:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f177461 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.465215:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f177461 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.465236:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f177461 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.465246:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f177461 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.465358:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067527680:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23564us (46015us total) trans 4294968073 rc 0/0 00000100:00100000:2.0:1789762935.465378:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 965 00000100:00100000:2.0:1789762935.465399:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408688655468 time 408688834669 state 0x2 ref 0 00000100:00100000:1.0:1789762935.469480:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.469554:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067527808 00000100:00100000:2.0:1789762935.469567:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067527808 00000100:00100000:2.0:1789762935.469623:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408688655468 time 408688834669 state 0x0 ref 1 00000100:00100000:2.0:1789762935.469641:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408738834669 time 408688834669 state 0x1 ref 1 00000100:00100000:2.0:1789762935.491835:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408738834669 time 408738898386 state 0x0 ref 1 00000100:00100000:2.0:1789762935.491885:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067527808:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762935.491989:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 44040192, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x15007 00010000:00010000:2.0:1789762935.492001:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177468 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.492021:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:2.0:1789762935.492027:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762935.492035:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177468 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.492067:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177468 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762935.515700:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968074 is committed 00002000:00010000:2.0:1789762935.515751:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 44040192 -> 45088768 00002000:00010000:2.0:1789762935.515761:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 86023 -> 88071 00010000:00010000:2.0:1789762935.515780:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177468 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.515806:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177468 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.515819:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177468 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.515828:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.515834:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177468 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.515875:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177468 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.515901:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177468 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.515911:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177468 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.516058:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067527808:12345-192.168.203.37@tcp:4:dd.0 Request processed in 24177us (46576us total) trans 4294968074 rc 0/0 00000100:00100000:2.0:1789762935.516074:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 966 00000100:00100000:2.0:1789762935.516091:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408738834669 time 408738898386 state 0x2 ref 0 00000100:00100000:1.0:1789762935.520235:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.520358:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067527936 00000100:00100000:2.0:1789762935.520374:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067527936 00000100:00100000:2.0:1789762935.520441:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408738834669 time 408738898386 state 0x0 ref 1 00000100:00100000:2.0:1789762935.520460:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408788898386 time 408738898386 state 0x1 ref 1 00000100:00100000:2.0:1789762935.542001:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408788898386 time 408789061530 state 0x0 ref 1 00000100:00100000:2.0:1789762935.542038:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067527936:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762935.542104:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 45088768, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x15807 00010000:00010000:2.0:1789762935.542116:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17746f lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.542137:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:2.0:1789762935.542144:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762935.542152:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17746f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.542170:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17746f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762935.563174:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968075 is committed 00002000:00010000:2.0:1789762935.563248:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 45088768 -> 46137344 00002000:00010000:2.0:1789762935.563256:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 88071 -> 90121 00010000:00010000:2.0:1789762935.563269:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17746f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.563282:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17746f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.563293:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17746f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.563301:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.563306:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17746f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.563315:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17746f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.563354:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17746f lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.563364:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f17746f lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.563468:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067527936:12345-192.168.203.37@tcp:4:dd.0 Request processed in 21442us (43237us total) trans 4294968075 rc 0/0 00000100:00100000:2.0:1789762935.563484:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 967 00000100:00100000:2.0:1789762935.563499:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408788898386 time 408789061530 state 0x2 ref 0 00000100:00100000:1.0:1789762935.567720:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.567864:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067528064 00000100:00100000:2.0:1789762935.567879:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067528064 00000100:00100000:2.0:1789762935.567997:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408788898386 time 408789061530 state 0x0 ref 1 00000100:00100000:2.0:1789762935.568035:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408839061530 time 408789061530 state 0x1 ref 1 00000100:00100000:3.0:1789762935.592165:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408839061530 time 408839225931 state 0x0 ref 1 00000100:00100000:3.0:1789762935.592198:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9629f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067528064:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762935.592257:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 46137344, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x16009 00010000:00010000:3.0:1789762935.592268:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177476 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.592287:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:3.0:1789762935.592293:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762935.592302:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177476 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.592327:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177476 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762935.616597:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968076 is committed 00002000:00010000:2.0:1789762935.616719:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 46137344 -> 47185920 00002000:00010000:2.0:1789762935.616728:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 90121 -> 92169 00010000:00010000:2.0:1789762935.616746:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177476 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.616764:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177476 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.616777:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177476 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.616786:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.616793:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177476 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.616805:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177476 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.616828:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177476 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.616864:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177476 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.617050:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9629f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067528064:12345-192.168.203.37@tcp:4:dd.0 Request processed in 24854us (49323us total) trans 4294968076 rc 0/0 00000100:00100000:2.0:1789762935.617071:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 968 00000100:00100000:2.0:1789762935.617087:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408839061530 time 408839225931 state 0x2 ref 0 00000100:00100000:0.0:1789762935.621288:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.621362:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067528192 00000100:00100000:2.0:1789762935.621375:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067528192 00000100:00100000:2.0:1789762935.621444:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408839061530 time 408839225931 state 0x0 ref 1 00000100:00100000:2.0:1789762935.621472:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408889225931 time 408839225931 state 0x1 ref 1 00000100:00100000:1.0:1789762935.642323:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408889225931 time 408889389169 state 0x0 ref 1 00000100:00100000:1.0:1789762935.642350:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067528192:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762935.642418:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 47185920, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x16809 00010000:00010000:1.0:1789762935.642429:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f17747d lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.642448:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:1.0:1789762935.642454:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762935.642462:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f17747d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.642481:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f17747d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762935.664170:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968077 is committed 00002000:00010000:2.0:1789762935.664298:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 47185920 -> 48234496 00002000:00010000:2.0:1789762935.664310:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 92169 -> 94217 00010000:00010000:2.0:1789762935.664330:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f17747d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.664356:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f17747d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.664376:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f17747d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.664391:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998fc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.664402:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f17747d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.664434:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f17747d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.664464:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f17747d lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.664482:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7a0ba2a/0xdf5a7ac59f17747d lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.664592:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067528192:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22242us (43301us total) trans 4294968077 rc 0/0 00000100:00100000:2.0:1789762935.664611:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 969 00000100:00100000:2.0:1789762935.664631:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408889225931 time 408889389169 state 0x2 ref 0 00000100:00100000:1.0:1789762935.668495:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.668623:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067528320 00000100:00100000:2.0:1789762935.668636:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067528320 00000100:00100000:2.0:1789762935.668695:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408889225931 time 408889389169 state 0x0 ref 1 00000100:00100000:2.0:1789762935.668714:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408939389169 time 408889389169 state 0x1 ref 1 00000100:00100000:1.0:1789762935.692535:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408939389169 time 408939598150 state 0x0 ref 1 00000100:00100000:1.0:1789762935.692564:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067528320:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762935.692620:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 48234496, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x17009 00010000:00010000:1.0:1789762935.692631:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177484 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.692652:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:1.0:1789762935.692658:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762935.692666:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177484 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.692682:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177484 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762935.714549:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968078 is committed 00002000:00010000:2.0:1789762935.714638:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 48234496 -> 49283072 00002000:00010000:2.0:1789762935.714647:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 94217 -> 96265 00010000:00010000:2.0:1789762935.714665:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177484 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.714691:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177484 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.714702:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177484 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.714711:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.714717:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177484 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.714729:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177484 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.714753:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177484 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.714769:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177484 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.714990:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067528320:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22427us (46491us total) trans 4294968078 rc 0/0 00000100:00100000:2.0:1789762935.715007:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 970 00000100:00100000:2.0:1789762935.715021:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408939389169 time 408939598150 state 0x2 ref 0 00000100:00100000:0.0:1789762935.719056:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.719119:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067528448 00000100:00100000:2.0:1789762935.719134:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067528448 00000100:00100000:2.0:1789762935.719197:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408939389169 time 408939598150 state 0x0 ref 1 00000100:00100000:2.0:1789762935.719215:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408989598150 time 408939598150 state 0x1 ref 1 00000100:00100000:1.0:1789762935.742685:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408989598150 time 408989746881 state 0x0 ref 1 00000100:00100000:1.0:1789762935.742713:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067528448:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762935.742774:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 49283072, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x17809 00010000:00010000:1.0:1789762935.742786:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17748b lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.742807:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=408s v=(0 0) 00010000:00010000:1.0:1789762935.742813:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762935.742821:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17748b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.742866:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17748b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762935.763893:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968079 is committed 00002000:00010000:2.0:1789762935.763989:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 49283072 -> 50331648 00002000:00010000:2.0:1789762935.764000:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 96265 -> 98313 00010000:00010000:2.0:1789762935.764018:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17748b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.764035:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17748b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.764046:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17748b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.764055:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.764061:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17748b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.764074:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17748b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.764099:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17748b lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.764108:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17748b lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.764212:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067528448:12345-192.168.203.37@tcp:4:dd.0 Request processed in 21504us (45157us total) trans 4294968079 rc 0/0 00000100:00100000:2.0:1789762935.764229:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 971 00000100:00100000:2.0:1789762935.764243:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408989598150 time 408989746881 state 0x2 ref 0 00000100:00100000:0.0:1789762935.768450:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.768574:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067528576 00000100:00100000:2.0:1789762935.768589:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067528576 00000100:00100000:2.0:1789762935.768649:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 408989598150 time 408989746881 state 0x0 ref 1 00000100:00100000:2.0:1789762935.768667:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409039746881 time 408989746881 state 0x1 ref 1 00000100:00100000:3.0:1789762935.792891:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409039746881 time 409039953892 state 0x0 ref 1 00000100:00100000:3.0:1789762935.792950:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067528576:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762935.793009:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 50331648, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x18009 00010000:00010000:3.0:1789762935.793020:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f177492 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.793039:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:3.0:1789762935.793047:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762935.793055:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f177492 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.793072:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f177492 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762935.816414:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968080 is committed 00002000:00010000:2.0:1789762935.816523:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 50331648 -> 51380224 00002000:00010000:2.0:1789762935.816535:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 98313 -> 100361 00010000:00010000:2.0:1789762935.816557:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f177492 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.816574:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f177492 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.816586:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f177492 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.816596:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762935.816602:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f177492 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.816623:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f177492 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.816649:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f177492 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762935.816658:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f177492 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762935.816775:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067528576:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23858us (48323us total) trans 4294968080 rc 0/0 00000100:00100000:2.0:1789762935.816790:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 972 00000100:00100000:2.0:1789762935.816806:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409039746881 time 409039953892 state 0x2 ref 0 00000100:00100000:1.0:1789762935.821010:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762935.821096:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067528704 00000100:00100000:3.0:1789762935.821110:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067528704 00000100:00100000:3.0:1789762935.821174:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409039746881 time 409039953892 state 0x0 ref 1 00000100:00100000:3.0:1789762935.821194:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409089953892 time 409039953892 state 0x1 ref 1 00000100:00100000:3.0:1789762935.842964:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409089953892 time 409090029579 state 0x0 ref 1 00000100:00100000:3.0:1789762935.842991:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067528704:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762935.843043:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 51380224, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x18809 00010000:00010000:3.0:1789762935.843054:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177499 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.843074:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:3.0:1789762935.843080:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762935.843087:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177499 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.843112:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177499 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762935.865617:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 51380224 -> 52428800 00002000:00010000:3.0:1789762935.865630:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 100361 -> 102409 00010000:00010000:3.0:1789762935.865651:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177499 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.865677:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177499 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.865695:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177499 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.865754:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762935.865764:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177499 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.865783:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177499 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.865814:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177499 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.865829:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177499 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762935.866020:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968081 is committed 00000100:00100000:3.0:1789762935.866034:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067528704:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23039us (45018us total) trans 4294968081 rc 0/0 00000100:00100000:3.0:1789762935.866059:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 973 00000100:00100000:3.0:1789762935.866079:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409089953892 time 409090029579 state 0x2 ref 0 00000100:00100000:1.0:1789762935.869748:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.869859:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067528832 00000100:00100000:2.0:1789762935.869874:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067528832 00000100:00100000:2.0:1789762935.869980:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409089953892 time 409090029579 state 0x0 ref 1 00000100:00100000:2.0:1789762935.870000:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409140029579 time 409090029579 state 0x1 ref 1 00000100:00100000:3.0:1789762935.893156:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409140029579 time 409140219873 state 0x0 ref 1 00000100:00100000:3.0:1789762935.893186:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9628000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067528832:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762935.893243:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 52428800, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x19009 00010000:00010000:3.0:1789762935.893254:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a0 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.893274:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:3.0:1789762935.893280:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762935.893288:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.893305:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762935.914543:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968082 is committed 00002000:00010000:3.0:1789762935.914603:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 52428800 -> 53477376 00002000:00010000:3.0:1789762935.914616:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 102409 -> 104457 00010000:00010000:3.0:1789762935.914638:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.914677:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.914696:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.914711:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762935.914720:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.914738:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.914763:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a0 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.914773:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a0 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762935.914914:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9628000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067528832:12345-192.168.203.37@tcp:4:dd.0 Request processed in 21728us (45159us total) trans 4294968082 rc 0/0 00000100:00100000:3.0:1789762935.914986:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 974 00000100:00100000:3.0:1789762935.915005:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409140029579 time 409140219873 state 0x2 ref 0 00000100:00100000:1.0:1789762935.919191:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762935.919266:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067528960 00000100:00100000:2.0:1789762935.919282:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067528960 00000100:00100000:2.0:1789762935.919347:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409140029579 time 409140219873 state 0x0 ref 1 00000100:00100000:2.0:1789762935.919373:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409190219873 time 409140219873 state 0x1 ref 1 00000100:00100000:3.0:1789762935.943346:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409190219873 time 409190412374 state 0x0 ref 1 00000100:00100000:3.0:1789762935.943374:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067528960:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762935.943427:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 53477376, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x19809 00010000:00010000:3.0:1789762935.943438:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a7 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.943456:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:3.0:1789762935.943463:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762935.943470:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.943487:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762935.964233:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968083 is committed 00002000:00010000:3.0:1789762935.964273:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 53477376 -> 54525952 00002000:00010000:3.0:1789762935.964287:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 104457 -> 106505 00010000:00010000:3.0:1789762935.964306:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.964331:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.964344:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.964354:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762935.964361:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.964373:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.964395:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a7 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.964404:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774a7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762935.964503:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067528960:12345-192.168.203.37@tcp:4:dd.0 Request processed in 21131us (45309us total) trans 4294968083 rc 0/0 00000100:00100000:3.0:1789762935.964522:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 975 00000100:00100000:3.0:1789762935.964541:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409190219873 time 409190412374 state 0x2 ref 0 00000100:00100000:1.0:1789762935.968359:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762935.968430:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067529088 00000100:00100000:3.0:1789762935.968443:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067529088 00000100:00100000:3.0:1789762935.968506:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409190219873 time 409190412374 state 0x0 ref 1 00000100:00100000:3.0:1789762935.968524:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409240412374 time 409190412374 state 0x1 ref 1 00000100:00100000:3.0:1789762935.993484:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409240412374 time 409240539785 state 0x0 ref 1 00000100:00100000:3.0:1789762935.993526:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067529088:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762935.993603:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 54525952, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x1a009 00010000:00010000:3.0:1789762935.993616:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774ae lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.993638:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:3.0:1789762935.993644:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762935.993652:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774ae lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762935.993671:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774ae lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762936.015468:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968084 is committed 00002000:00010000:3.0:1789762936.015571:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 54525952 -> 55574528 00002000:00010000:3.0:1789762936.015582:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 106505 -> 108553 00010000:00010000:3.0:1789762936.015601:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774ae lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.015622:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774ae lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.015635:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774ae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.015646:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762936.015652:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774ae lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.015674:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774ae lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.015697:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774ae lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.015707:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774ae lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762936.015801:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067529088:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22291us (47442us total) trans 4294968084 rc 0/0 00000100:00100000:3.0:1789762936.015815:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 976 00000100:00100000:3.0:1789762936.015830:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409240412374 time 409240539785 state 0x2 ref 0 00000100:00100000:0.0:1789762936.020241:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762936.020318:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067529216 00000100:00100000:3.0:1789762936.020332:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067529216 00000100:00100000:3.0:1789762936.020391:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409240412374 time 409240539785 state 0x0 ref 1 00000100:00100000:3.0:1789762936.020410:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409290539785 time 409240539785 state 0x1 ref 1 00000100:00100000:3.0:1789762936.043551:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409290539785 time 409290617156 state 0x0 ref 1 00000100:00100000:3.0:1789762936.043580:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067529216:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762936.043631:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 55574528, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x1a809 00010000:00010000:3.0:1789762936.043647:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774b5 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.043672:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:3.0:1789762936.043680:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762936.043692:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774b5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.043718:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774b5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762936.066281:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968085 is committed 00002000:00010000:3.0:1789762936.066325:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 55574528 -> 56623104 00002000:00010000:3.0:1789762936.066336:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 108553 -> 110601 00010000:00010000:3.0:1789762936.066353:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774b5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.066372:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774b5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.066384:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774b5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.066393:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762936.066400:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774b5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.066412:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774b5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.066437:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774b5 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.066448:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774b5 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762936.066550:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067529216:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22975us (46310us total) trans 4294968085 rc 0/0 00000100:00100000:3.0:1789762936.066562:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 977 00000100:00100000:3.0:1789762936.066578:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409290539785 time 409290617156 state 0x2 ref 0 00000100:00100000:0.0:1789762936.070500:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762936.070567:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067529344 00000100:00100000:3.0:1789762936.070579:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067529344 00000100:00100000:3.0:1789762936.070641:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409290539785 time 409290617156 state 0x0 ref 1 00000100:00100000:3.0:1789762936.070668:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409340617156 time 409290617156 state 0x1 ref 1 00000100:00100000:2.0:1789762936.093724:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409340617156 time 409340781887 state 0x0 ref 1 00000100:00100000:2.0:1789762936.093763:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067529344:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762936.093875:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 56623104, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x1b009 00010000:00010000:2.0:1789762936.093888:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1774bc lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.093911:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:2.0:1789762936.093957:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762936.093971:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1774bc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.093999:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1774bc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762936.116240:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968086 is committed 00002000:00010000:3.0:1789762936.116279:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 56623104 -> 57671680 00002000:00010000:3.0:1789762936.116292:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 110601 -> 112649 00010000:00010000:3.0:1789762936.116313:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1774bc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.116330:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1774bc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.116351:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1774bc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.116361:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762936.116367:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1774bc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.116378:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1774bc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.116403:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1774bc lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.116413:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1774bc lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762936.116517:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067529344:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22765us (46020us total) trans 4294968086 rc 0/0 00000100:00100000:3.0:1789762936.116532:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 978 00000100:00100000:3.0:1789762936.116549:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409340617156 time 409340781887 state 0x2 ref 0 00000100:00100000:1.0:1789762936.121133:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762936.121202:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067529472 00000100:00100000:3.0:1789762936.121216:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067529472 00000100:00100000:3.0:1789762936.121279:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409340617156 time 409340781887 state 0x0 ref 1 00000100:00100000:3.0:1789762936.121305:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409390781887 time 409340781887 state 0x1 ref 1 00000100:00100000:2.0:1789762936.143878:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409390781887 time 409390939001 state 0x0 ref 1 00000100:00100000:2.0:1789762936.143906:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067529472:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762936.143993:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 57671680, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x1b809 00010000:00010000:2.0:1789762936.144005:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f1774c3 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.144025:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:2.0:1789762936.144031:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762936.144039:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f1774c3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.144056:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f1774c3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762936.173557:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968087 is committed 00002000:00010000:3.0:1789762936.173635:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 57671680 -> 58720256 00002000:00010000:3.0:1789762936.173645:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 112649 -> 114697 00010000:00010000:3.0:1789762936.173665:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f1774c3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.173684:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f1774c3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.173696:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f1774c3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.173706:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762936.173712:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f1774c3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.173725:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f1774c3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.173748:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f1774c3 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.173758:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f1774c3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762936.173976:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9629180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067529472:12345-192.168.203.37@tcp:4:dd.0 Request processed in 30072us (52840us total) trans 4294968087 rc 0/0 00000100:00100000:3.0:1789762936.173995:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 979 00000100:00100000:3.0:1789762936.174013:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409390781887 time 409390939001 state 0x2 ref 0 00000100:00100000:0.0:1789762936.178601:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762936.178665:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067529600 00000100:00100000:3.0:1789762936.178680:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067529600 00000100:00100000:3.0:1789762936.178744:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409390781887 time 409390939001 state 0x0 ref 1 00000100:00100000:3.0:1789762936.178764:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409440939001 time 409390939001 state 0x1 ref 1 00000100:00100000:2.0:1789762936.194033:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409440939001 time 409441097502 state 0x0 ref 1 00000100:00100000:2.0:1789762936.194072:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067529600:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762936.194142:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 58720256, mtime: 0x6aad9d77, atime: 0x0, ctime: 0x6aad9d77, blocks: 0x1c009 00010000:00010000:2.0:1789762936.194158:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774ca lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.194183:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:2.0:1789762936.194190:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762936.194198:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774ca lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.194217:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774ca lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762936.216231:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968088 is committed 00002000:00010000:3.0:1789762936.216252:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 58720256 -> 59768832 00002000:00010000:3.0:1789762936.216262:0:13300:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb mtime from disk: 1789762935 -> 1789762936 00002000:00010000:3.0:1789762936.216267:0:13300:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb ctime from disk: 1789762935 -> 1789762936 00002000:00010000:3.0:1789762936.216271:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 114697 -> 116745 00010000:00010000:3.0:1789762936.216288:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774ca lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.216304:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774ca lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.216317:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774ca lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.216327:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762936.216335:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774ca lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.216348:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774ca lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.216372:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774ca lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.216381:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774ca lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762936.216486:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067529600:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22424us (37884us total) trans 4294968088 rc 0/0 00000100:00100000:3.0:1789762936.216501:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 980 00000100:00100000:3.0:1789762936.216517:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409440939001 time 409441097502 state 0x2 ref 0 00000100:00100000:1.0:1789762936.220816:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762936.220899:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067529728 00000100:00100000:3.0:1789762936.220913:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067529728 00000100:00100000:3.0:1789762936.221038:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409440939001 time 409441097502 state 0x0 ref 1 00000100:00100000:3.0:1789762936.221056:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409491097502 time 409441097502 state 0x1 ref 1 00000100:00100000:2.0:1789762936.244196:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409491097502 time 409491261919 state 0x0 ref 1 00000100:00100000:2.0:1789762936.244225:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9628a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067529728:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762936.244293:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 59768832, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x1c809 00010000:00010000:2.0:1789762936.244304:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774d1 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.244321:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:2.0:1789762936.244345:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762936.244357:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774d1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.244383:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774d1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762936.266083:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968089 is committed 00002000:00010000:3.0:1789762936.266188:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 59768832 -> 60817408 00002000:00010000:3.0:1789762936.266198:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 116745 -> 118793 00010000:00010000:3.0:1789762936.266213:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774d1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.266231:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774d1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.266242:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774d1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.266252:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafa00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762936.266257:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774d1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.266269:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774d1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.266291:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774d1 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.266300:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774d1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762936.266403:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9628a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067529728:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22182us (45586us total) trans 4294968089 rc 0/0 00000100:00100000:3.0:1789762936.266417:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 981 00000100:00100000:3.0:1789762936.266431:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409491097502 time 409491261919 state 0x2 ref 0 00000100:00100000:1.0:1789762936.270269:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762936.270328:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067529856 00000100:00100000:3.0:1789762936.270342:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067529856 00000100:00100000:3.0:1789762936.270400:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409491097502 time 409491261919 state 0x0 ref 1 00000100:00100000:3.0:1789762936.270417:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409541261919 time 409491261919 state 0x1 ref 1 00000100:00100000:2.0:1789762936.294361:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409541261919 time 409541413900 state 0x0 ref 1 00000100:00100000:2.0:1789762936.294408:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067529856:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762936.294488:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 60817408, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x1d009 00010000:00010000:2.0:1789762936.294507:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774d8 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.294548:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:2.0:1789762936.294558:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762936.294571:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774d8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.294594:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774d8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762936.317585:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968090 is committed 00002000:00010000:3.0:1789762936.317717:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 60817408 -> 61865984 00002000:00010000:3.0:1789762936.317726:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 118793 -> 120841 00010000:00010000:3.0:1789762936.317743:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774d8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.317761:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774d8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.317773:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774d8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.317783:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762936.317789:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774d8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.317801:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774d8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.317825:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774d8 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.317862:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774d8 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762936.318030:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067529856:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23634us (47756us total) trans 4294968090 rc 0/0 00000100:00100000:3.0:1789762936.318047:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 982 00000100:00100000:3.0:1789762936.318064:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409541261919 time 409541413900 state 0x2 ref 0 00000100:00100000:0.0:1789762936.320186:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074742144:0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.320268:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074742016:0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.320287:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762936.320367:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762936.320381:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074741888:0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.320400:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074742656:0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.320430:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e81f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074742528:0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.320480:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762936.320482:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762936.320509:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762936.320534:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e83b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074743168:0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.320586:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e82680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074743040:0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.320603:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762936.320617:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb452a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074742400:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762936.320649:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762936.320663:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eb2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074743680:0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.320703:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074743552:0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.320718:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762936.320740:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762936.320789:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074742144 00000100:00100000:2.0:1789762936.320799:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074742144 00000100:00100000:1.0:1789762936.320860:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e83800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074742784:0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.320892:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074742656 00000100:00100000:2.0:1789762936.320899:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074742656 00000100:00100000:2.0:1789762936.320985:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 419 00000100:00100000:1.0:1789762936.321000:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762936.321001:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9628380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074742272:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762936.321010:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074742144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.321063:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fdfe0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074743296:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762936.321077:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074742016 00000100:00100000:0.0:1789762936.321090:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074742016 00000100:00100000:3.0:1789762936.321091:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762936.321121:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074742144:12345-0@lo:400:kworker.0 Request processed in 119us (840us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762936.321134:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 419 00000100:00100000:1.0:1789762936.321146:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762936.321156:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fdfe0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074742912:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762936.321164:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074742528 00000100:00100000:0.0:1789762936.321171:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074742528 00000100:00100000:2.0:1789762936.321177:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 420 00000100:00100000:2.0:1789762936.321195:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074742656:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.321213:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eb0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074743808:0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.321220:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074743168 00000100:00100000:3.0:1789762936.321228:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762936.321231:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074743168 00000100:00100000:2.0:1789762936.321261:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074742656:12345-0@lo:400:kworker.0 Request processed in 69us (782us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762936.321274:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 420 00000100:00100000:0.0:1789762936.321292:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074743040 00000100:00100000:3.0:1789762936.321292:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e82300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074743424:0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.321303:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762936.321304:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074743040 00000100:00100000:2.0:1789762936.321315:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074742784 00000100:00100000:2.0:1789762936.321327:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074742784 00000100:00100000:0.0:1789762936.321362:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 622 00000100:00100000:3.0:1789762936.321370:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762936.321383:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c9ffa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074742016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.321398:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074743424 00000100:00100000:2.0:1789762936.321409:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074743424 00000100:00100000:3.0:1789762936.321412:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eb1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074743936:0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.321434:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074742400 00000100:00100000:1.0:1789762936.321447:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074742400 00000100:00100000:0.0:1789762936.321469:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c9ffa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074742016:12345-0@lo:400:kworker.0 Request processed in 88us (1105us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762936.321477:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 623 00000100:00100000:3.0:1789762936.321479:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762936.321497:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 622 00000100:00100000:2.0:1789762936.321509:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074742528:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.321536:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074742272 00000100:00100000:0.0:1789762936.321545:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 624 00000100:00100000:1.0:1789762936.321552:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074742272 00000100:00100000:3.0:1789762936.321560:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074741888 00000100:00100000:0.0:1789762936.321568:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074743168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762936.321572:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074741888 00000100:00100000:2.0:1789762936.321596:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3427:x1876702074742528:12345-0@lo:400:kworker.0 Request processed in 97us (1090us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762936.321611:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 623 00000100:00100000:1.0:1789762936.321615:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074743296 00000100:00100000:1.0:1789762936.321626:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074743296 00000100:00100000:2.0:1789762936.321648:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 625 00000100:00100000:3.0:1789762936.321658:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:0.0:1789762936.321659:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074743168:12345-0@lo:400:kworker.0 Request processed in 96us (1059us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762936.321671:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074742912 00000100:00100000:2.0:1789762936.321672:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074743040:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.321684:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074742912 00000100:00100000:0.0:1789762936.321685:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 624 00000100:00100000:3.0:1789762936.321687:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15c5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074741888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.321734:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 626 00000100:00100000:1.0:1789762936.321762:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 254 00000100:00100000:0.0:1789762936.321764:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9628e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074742784:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.321767:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074743040:12345-0@lo:400:kworker.0 Request processed in 104us (1118us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762936.321799:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 625 00000100:00100000:1.0:1789762936.321800:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074742400:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762936.321801:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15c5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074741888:12345-0@lo:400:kworker.0 Request processed in 116us (1305us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762936.321827:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 627 00000100:00100000:3.0:1789762936.321866:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00100000:2.0:1789762936.321870:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074743424:12345-0@lo:400:kworker.0 00002000:00000024:1.0:1789762936.321880:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526712832 pending 0 free 3830448128 avail 3795845120 00000100:00100000:0.0:1789762936.321895:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9628e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3425:x1876702074742784:12345-0@lo:400:kworker.0 Request processed in 143us (901us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762936.321911:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 626 00000100:00100000:3.0:1789762936.321993:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 255 00000100:00100000:2.0:1789762936.322003:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074743424:12345-0@lo:400:kworker.0 Request processed in 136us (632us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762936.322016:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 256 00000100:00100000:3.0:1789762936.322020:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074742272:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762936.322028:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 627 00000100:00100000:1.0:1789762936.322040:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074742400:12345-0@lo:13:osp-pre-0-1.0 Request processed in 249us (1324us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762936.322041:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9629500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074743296:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789762936.322059:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526712832 pending 0 free 3830448128 avail 3795845120 00000100:00100000:1.0:1789762936.322079:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 254 00002000:00000024:0.0:1789762936.322091:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762936.322097:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074743680 00000100:00100000:2.0:1789762936.322106:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074743680 00000100:00100000:1.0:1789762936.322135:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 257 00000100:00100000:3.0:1789762936.322147:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074742272:12345-0@lo:13:osp-pre-0-0.0 Request processed in 131us (1052us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762936.322166:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074743808 00000100:00100000:1.0:1789762936.322167:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074742912:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762936.322168:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 255 00000100:00100000:0.0:1789762936.322171:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9629500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074743296:12345-0@lo:13:osp-pre-1-0.0 Request processed in 140us (1030us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762936.322177:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074743808 00000100:00100000:0.0:1789762936.322198:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 256 00002000:00000024:1.0:1789762936.322214:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762936.322231:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074743936 00000100:00100000:2.0:1789762936.322238:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074743936 00000100:00100000:3.0:1789762936.322246:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 255 00000100:00100000:3.0:1789762936.322272:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074743680:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.322284:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 256 00000100:00100000:1.0:1789762936.322297:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074742912:12345-0@lo:13:osp-pre-1-1.0 Request processed in 143us (1072us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762936.322307:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074743808:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.322320:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 257 00000100:00100000:3.0:1789762936.322371:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074743680:12345-0@lo:400:kworker.0 Request processed in 106us (1634us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762936.322377:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762936.322386:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 255 00000100:00080000:1.0:1789762936.322393:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789762936.322403:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762936.322407:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9629880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074743808:12345-0@lo:400:kworker.0 Request processed in 104us (1107us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762936.322412:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762936.322425:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 257 00000100:00100000:2.0:1789762936.322438:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 256 00000100:00100000:3.0:1789762936.322446:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074743936:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762936.322505:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074743936:12345-0@lo:400:kworker.0 Request processed in 67us (1032us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762936.322519:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 257 00000100:00100000:2.0:1789762936.322559:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762936.322612:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074741888:0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.322647:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074742016:0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.322651:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb452a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074742400:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762936.322671:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9628380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074742272:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762936.322691:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e81f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3427:1876702074742528:0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.322710:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e83800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3425:1876702074742784:0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.322861:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e82680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074743040:0@lo:400:kworker.0 00000100:00100000:3.0:1789762936.322871:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fdfe0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074742912:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762936.322876:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fdfe0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074743296:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762936.322970:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074743552 00000100:00100000:3.0:1789762936.322974:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e82300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074743424:0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.322979:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074743552 00000100:00100000:1.0:1789762936.322992:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eb0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074743808:0@lo:400:kworker.0 00000100:00100000:3.0:1789762936.323028:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eb1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074743936:0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.323035:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762936.323047:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 258 00000100:00100000:2.0:1789762936.323066:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eafb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074743552:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.323108:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067529984 00000100:00100000:0.0:1789762936.323120:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067529984 00000100:00100000:2.0:1789762936.323140:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eafb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074743552:12345-0@lo:400:kworker.0 Request processed in 79us (587us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762936.323151:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 258 00000100:00100000:0.0:1789762936.323189:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409541261919 time 409541413900 state 0x0 ref 1 00000100:00100000:3.0:1789762936.323196:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074742656:0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.323215:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074743552:0@lo:400:kworker.0 00000100:00100000:3.0:1789762936.323231:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e83b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074743168:0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.323232:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409591413900 time 409541413900 state 0x1 ref 1 00000100:00100000:3.0:1789762936.323264:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eb2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074743680:0@lo:400:kworker.0 00000100:00100000:3.0:1789762936.323288:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074742144:0@lo:400:kworker.0 00000100:00100000:3.0:1789762936.344465:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409591413900 time 409591531795 state 0x0 ref 1 00000100:00100000:3.0:1789762936.344490:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067529984:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762936.344538:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 61865984, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x1d809 00010000:00010000:3.0:1789762936.344549:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774df lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.344569:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:3.0:1789762936.344575:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762936.344582:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774df lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.344599:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774df lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762936.366531:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968091 is committed 00002000:00010000:3.0:1789762936.366566:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 61865984 -> 62914560 00002000:00010000:3.0:1789762936.366576:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 120841 -> 122889 00010000:00010000:3.0:1789762936.366591:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774df lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.366607:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774df lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.366628:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774df lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.366637:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762936.366643:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774df lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.366654:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774df lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.366675:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774df lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.366685:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f1774df lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762936.366777:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067529984:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22290us (43748us total) trans 4294968091 rc 0/0 00000100:00100000:3.0:1789762936.366791:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 983 00000100:00100000:3.0:1789762936.366805:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409591413900 time 409591531795 state 0x2 ref 0 00000100:00100000:0.0:1789762936.370761:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762936.370959:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067530112 00000100:00100000:3.0:1789762936.370972:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067530112 00000100:00100000:3.0:1789762936.371030:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409591413900 time 409591531795 state 0x0 ref 1 00000100:00100000:3.0:1789762936.371055:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409641531795 time 409591531795 state 0x1 ref 1 00000100:00100000:2.0:1789762936.394602:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409641531795 time 409641663822 state 0x0 ref 1 00000100:00100000:2.0:1789762936.394635:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067530112:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762936.394697:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 62914560, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x1e009 00010000:00010000:2.0:1789762936.394709:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774e6 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.394730:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:2.0:1789762936.394736:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762936.394744:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774e6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.394762:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774e6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762936.415350:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968092 is committed 00002000:00010000:3.0:1789762936.415421:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 62914560 -> 63963136 00002000:00010000:3.0:1789762936.415433:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 122889 -> 124937 00010000:00010000:3.0:1789762936.415453:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774e6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.415475:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774e6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.415492:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774e6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.415506:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762936.415515:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774e6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.415532:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774e6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.415563:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774e6 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.415580:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f1774e6 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762936.415699:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067530112:12345-192.168.203.37@tcp:4:dd.0 Request processed in 21072us (44938us total) trans 4294968092 rc 0/0 00000100:00100000:3.0:1789762936.415713:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 984 00000100:00100000:3.0:1789762936.415729:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409641531795 time 409641663822 state 0x2 ref 0 00000100:00100000:0.0:1789762936.419973:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762936.420040:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067530240 00000100:00100000:3.0:1789762936.420055:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067530240 00000100:00100000:3.0:1789762936.420114:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409641531795 time 409641663822 state 0x0 ref 1 00000100:00100000:3.0:1789762936.420132:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409691663822 time 409641663822 state 0x1 ref 1 00000100:00100000:2.0:1789762936.444731:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409691663822 time 409691797180 state 0x0 ref 1 00000100:00100000:2.0:1789762936.444757:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067530240:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762936.444806:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 63963136, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x1e809 00010000:00010000:2.0:1789762936.444817:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774ed lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.444858:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:2.0:1789762936.444865:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762936.444873:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774ed lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.444891:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774ed lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.448256:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762936.466523:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968093 is committed 00002000:00010000:3.0:1789762936.466549:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 63963136 -> 65011712 00002000:00010000:3.0:1789762936.466558:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 124937 -> 126985 00010000:00010000:3.0:1789762936.466573:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774ed lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.466590:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774ed lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.466602:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774ed lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.466611:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafa00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762936.466617:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774ed lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.466637:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774ed lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.466658:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774ed lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.466668:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f1774ed lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762936.466758:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067530240:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22006us (46788us total) trans 4294968093 rc 0/0 00000100:00100000:3.0:1789762936.466770:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 985 00000100:00100000:3.0:1789762936.466783:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409691663822 time 409691797180 state 0x2 ref 0 00000100:00100000:0.0:1789762936.470696:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762936.470762:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067530368 00000100:00100000:3.0:1789762936.470775:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067530368 00000100:00100000:3.0:1789762936.470853:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409691663822 time 409691797180 state 0x0 ref 1 00000100:00100000:3.0:1789762936.470878:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409741797180 time 409691797180 state 0x1 ref 1 00000100:00100000:2.0:1789762936.494948:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409741797180 time 409741983797 state 0x0 ref 1 00000100:00100000:2.0:1789762936.494979:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067530368:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762936.495037:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 65011712, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x1f009 00010000:00010000:2.0:1789762936.495048:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774f4 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.495068:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:2.0:1789762936.495074:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762936.495081:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774f4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.495099:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774f4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762936.518305:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968094 is committed 00002000:00010000:2.0:1789762936.518331:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 65011712 -> 66060288 00002000:00010000:2.0:1789762936.518343:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 126985 -> 129033 00010000:00010000:2.0:1789762936.518361:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774f4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.518380:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774f4 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.518400:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774f4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.518409:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762936.518415:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774f4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.518427:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774f4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.518450:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774f4 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.518460:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f1774f4 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762936.518585:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067530368:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23610us (47887us total) trans 4294968094 rc 0/0 00000100:00100000:2.0:1789762936.518601:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 986 00000100:00100000:2.0:1789762936.518618:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409741797180 time 409741983797 state 0x2 ref 0 00000100:00100000:0.0:1789762936.523020:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762936.523122:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067530496 00000100:00100000:3.0:1789762936.523141:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067530496 00000100:00100000:3.0:1789762936.523229:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409741797180 time 409741983797 state 0x0 ref 1 00000100:00100000:3.0:1789762936.523252:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409791983797 time 409741983797 state 0x1 ref 1 00000100:00100000:0.0:1789762936.545055:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409791983797 time 409792114056 state 0x0 ref 1 00000100:00100000:0.0:1789762936.545090:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067530496:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:0.0:1789762936.545164:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 66060288, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x1f809 00010000:00010000:0.0:1789762936.545174:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f1774fb lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.545197:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:0.0:1789762936.545202:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762936.545208:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f1774fb lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.545223:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f1774fb lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762936.567306:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968095 is committed 00002000:00010000:2.0:1789762936.567438:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 66060288 -> 67108864 00002000:00010000:2.0:1789762936.567449:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 129033 -> 131081 00010000:00010000:2.0:1789762936.567471:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f1774fb lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.567498:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f1774fb lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.567510:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f1774fb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.567519:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a6400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762936.567526:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f1774fb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.567536:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f1774fb lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.567561:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f1774fb lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.567571:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000468f495d/0xdf5a7ac59f1774fb lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762936.567696:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067530496:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22614us (44674us total) trans 4294968095 rc 0/0 00000100:00100000:2.0:1789762936.567712:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 987 00000100:00100000:2.0:1789762936.567730:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409791983797 time 409792114056 state 0x2 ref 0 00000100:00100000:0.0:1789762936.571901:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762936.571992:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067530624 00000100:00100000:3.0:1789762936.572007:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067530624 00000100:00100000:3.0:1789762936.572073:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409791983797 time 409792114056 state 0x0 ref 1 00000100:00100000:3.0:1789762936.572092:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409842114056 time 409792114056 state 0x1 ref 1 00000100:00100000:2.0:1789762936.595193:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409842114056 time 409842249856 state 0x0 ref 1 00000100:00100000:2.0:1789762936.595232:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067530624:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762936.595300:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 67108864, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x20009 00010000:00010000:2.0:1789762936.595312:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177502 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.595349:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:2.0:1789762936.595355:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762936.595364:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177502 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.595383:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177502 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762936.616023:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968096 is committed 00002000:00010000:2.0:1789762936.616133:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 67108864 -> 68157440 00002000:00010000:2.0:1789762936.616144:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 131081 -> 133129 00010000:00010000:2.0:1789762936.616164:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177502 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.616189:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177502 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.616207:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177502 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.616222:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762936.616230:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177502 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.616243:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177502 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.616265:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177502 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.616275:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177502 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762936.616371:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067530624:12345-192.168.203.37@tcp:4:dd.0 Request processed in 21150us (44475us total) trans 4294968096 rc 0/0 00000100:00100000:2.0:1789762936.616385:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 988 00000100:00100000:2.0:1789762936.616400:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409842114056 time 409842249856 state 0x2 ref 0 00000100:00100000:1.0:1789762936.620302:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762936.620373:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067530752 00000100:00100000:2.0:1789762936.620387:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067530752 00000100:00100000:2.0:1789762936.620454:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409842114056 time 409842249856 state 0x0 ref 1 00000100:00100000:2.0:1789762936.620473:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409892249856 time 409842249856 state 0x1 ref 1 00000100:00100000:2.0:1789762936.645276:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409892249856 time 409892337153 state 0x0 ref 1 00000100:00100000:2.0:1789762936.645322:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9629c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067530752:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762936.645387:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 68157440, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x20809 00010000:00010000:2.0:1789762936.645399:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177509 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.645419:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:2.0:1789762936.645425:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762936.645433:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177509 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.645452:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177509 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762936.668170:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968097 is committed 00002000:00010000:2.0:1789762936.668295:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 68157440 -> 69206016 00002000:00010000:2.0:1789762936.668305:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 133129 -> 135177 00010000:00010000:2.0:1789762936.668325:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177509 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.668349:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177509 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.668367:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177509 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.668382:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762936.668389:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177509 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.668409:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177509 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.668432:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177509 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.668442:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177509 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762936.668541:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9629c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067530752:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23240us (48239us total) trans 4294968097 rc 0/0 00000100:00100000:2.0:1789762936.668555:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 989 00000100:00100000:2.0:1789762936.668570:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409892249856 time 409892337153 state 0x2 ref 0 00000100:00100000:1.0:1789762936.672815:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762936.672898:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067530880 00000100:00100000:2.0:1789762936.672913:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067530880 00000100:00100000:2.0:1789762936.673013:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409892249856 time 409892337153 state 0x0 ref 1 00000100:00100000:2.0:1789762936.673034:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409942337153 time 409892337153 state 0x1 ref 1 00000100:00100000:2.0:1789762936.695378:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409942337153 time 409942438327 state 0x0 ref 1 00000100:00100000:2.0:1789762936.695412:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067530880:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762936.695481:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 69206016, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x21009 00010000:00010000:2.0:1789762936.695494:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177510 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.695515:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:2.0:1789762936.695521:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762936.695529:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177510 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.695548:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177510 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762936.718474:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968098 is committed 00002000:00010000:2.0:1789762936.718532:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 69206016 -> 70254592 00002000:00010000:2.0:1789762936.718545:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 135177 -> 137225 00010000:00010000:2.0:1789762936.718566:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177510 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.718591:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177510 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.718610:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177510 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.718625:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762936.718631:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177510 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.718644:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177510 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.718670:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177510 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.718680:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177510 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762936.718787:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067530880:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23382us (45970us total) trans 4294968098 rc 0/0 00000100:00100000:2.0:1789762936.718800:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 990 00000100:00100000:2.0:1789762936.718815:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409942337153 time 409942438327 state 0x2 ref 0 00000100:00100000:0.0:1789762936.722656:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762936.722791:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067531008 00000100:00100000:2.0:1789762936.722806:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067531008 00000100:00100000:2.0:1789762936.722901:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409942337153 time 409942438327 state 0x0 ref 1 00000100:00100000:2.0:1789762936.722964:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409992438327 time 409942438327 state 0x1 ref 1 00000100:00100000:3.0:1789762936.745596:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409992438327 time 409992654348 state 0x0 ref 1 00000100:00100000:3.0:1789762936.745625:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067531008:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762936.745679:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 70254592, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x21809 00010000:00010000:3.0:1789762936.745690:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177517 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.745709:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=409s v=(0 0) 00010000:00010000:3.0:1789762936.745716:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762936.745723:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177517 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.745741:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177517 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762936.767591:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968099 is committed 00002000:00010000:2.0:1789762936.767629:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 70254592 -> 71303168 00002000:00010000:2.0:1789762936.767640:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 137225 -> 139273 00010000:00010000:2.0:1789762936.767661:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177517 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.767686:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177517 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.767719:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177517 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.767735:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762936.767744:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177517 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.767763:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177517 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.767795:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177517 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.767811:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177517 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762936.767973:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067531008:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22303us (45268us total) trans 4294968099 rc 0/0 00000100:00100000:2.0:1789762936.767992:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 991 00000100:00100000:2.0:1789762936.768012:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409992438327 time 409992654348 state 0x2 ref 0 00000100:00100000:0.0:1789762936.771381:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762936.771498:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067531136 00000100:00100000:2.0:1789762936.771515:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067531136 00000100:00100000:2.0:1789762936.771587:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 409992438327 time 409992654348 state 0x0 ref 1 00000100:00100000:2.0:1789762936.771623:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410042654348 time 409992654348 state 0x1 ref 1 00000100:00100000:3.0:1789762936.795736:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410042654348 time 410042798689 state 0x0 ref 1 00000100:00100000:3.0:1789762936.795767:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067531136:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762936.795826:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 71303168, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x22009 00010000:00010000:3.0:1789762936.795863:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f17751e lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.795883:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762936.795890:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762936.795898:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f17751e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.795946:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f17751e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762936.818226:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 71303168 -> 72351744 00002000:00010000:3.0:1789762936.818236:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 139273 -> 141321 00000001:00080000:2.0:1789762936.818252:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968100 is committed 00010000:00010000:3.0:1789762936.818253:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f17751e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.818275:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f17751e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.818288:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f17751e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.818298:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762936.818305:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f17751e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.818325:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f17751e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.818347:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f17751e lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.818357:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f17751e lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762936.818450:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067531136:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22690us (47068us total) trans 4294968100 rc 0/0 00000100:00100000:3.0:1789762936.818463:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 992 00000100:00100000:3.0:1789762936.818478:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410042654348 time 410042798689 state 0x2 ref 0 00000100:00100000:1.0:1789762936.822087:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762936.822182:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067531264 00000100:00100000:2.0:1789762936.822195:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067531264 00000100:00100000:2.0:1789762936.822254:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410042654348 time 410042798689 state 0x0 ref 1 00000100:00100000:2.0:1789762936.822272:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410092798689 time 410042798689 state 0x1 ref 1 00000100:00100000:3.0:1789762936.845973:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410092798689 time 410093033883 state 0x0 ref 1 00000100:00100000:3.0:1789762936.846008:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9628700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067531264:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762936.846065:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 72351744, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x22809 00010000:00010000:3.0:1789762936.846081:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177525 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.846099:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762936.846105:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762936.846113:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177525 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.846130:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177525 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762936.868150:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 72351744 -> 73400320 00002000:00010000:2.0:1789762936.868160:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 141321 -> 143369 00010000:00010000:2.0:1789762936.868175:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177525 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.868192:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177525 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.868203:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177525 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.868212:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762936.868218:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177525 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.868229:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177525 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.868259:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177525 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.868270:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ddaca193/0xdf5a7ac59f177525 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762936.868378:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9628700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067531264:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22375us (46288us total) trans 4294968101 rc 0/0 00000100:00100000:2.0:1789762936.868397:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 993 00000100:00100000:2.0:1789762936.868418:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410092798689 time 410093033883 state 0x2 ref 0 00000001:00080000:2.0:1789762936.868480:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968101 is committed 00000100:00100000:1.0:1789762936.872995:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762936.873056:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067531392 00000100:00100000:2.0:1789762936.873069:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067531392 00000100:00100000:2.0:1789762936.873126:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410092798689 time 410093033883 state 0x0 ref 1 00000100:00100000:2.0:1789762936.873144:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410143033883 time 410093033883 state 0x1 ref 1 00000100:00100000:3.0:1789762936.896147:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410143033883 time 410143204328 state 0x0 ref 1 00000100:00100000:3.0:1789762936.896188:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067531392:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762936.896262:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 73400320, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x23009 00010000:00010000:3.0:1789762936.896274:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f17752c lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.896295:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762936.896301:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762936.896310:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f17752c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.896329:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f17752c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762936.920144:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968102 is committed 00002000:00010000:2.0:1789762936.920318:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 73400320 -> 74448896 00002000:00010000:2.0:1789762936.920330:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 143369 -> 145417 00010000:00010000:2.0:1789762936.920353:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f17752c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.920371:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f17752c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.920392:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f17752c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.920402:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762936.920408:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f17752c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.920420:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f17752c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.920444:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f17752c lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.920454:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f0d46e9b/0xdf5a7ac59f17752c lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762936.920568:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067531392:12345-192.168.203.37@tcp:4:dd.0 Request processed in 24393us (47572us total) trans 4294968102 rc 0/0 00000100:00100000:2.0:1789762936.920582:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 994 00000100:00100000:2.0:1789762936.920598:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410143033883 time 410143204328 state 0x2 ref 0 00000100:00100000:0.0:1789762936.925129:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762936.925217:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067531520 00000100:00100000:3.0:1789762936.925232:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067531520 00000100:00100000:3.0:1789762936.925299:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410143033883 time 410143204328 state 0x0 ref 1 00000100:00100000:3.0:1789762936.925318:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410193204328 time 410143204328 state 0x1 ref 1 00000100:00100000:3.0:1789762936.946224:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410193204328 time 410193290295 state 0x0 ref 1 00000100:00100000:3.0:1789762936.946250:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067531520:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762936.946301:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 74448896, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x23809 00010000:00010000:3.0:1789762936.946312:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177533 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.946332:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762936.946338:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762936.946345:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177533 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.946363:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177533 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762936.967975:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968103 is committed 00002000:00010000:2.0:1789762936.968005:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:2.0:1789762936.968014:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 145417 -> 147465 00010000:00010000:2.0:1789762936.968030:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177533 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.968055:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177533 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.968067:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177533 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.968076:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762936.968081:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177533 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.968093:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177533 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.968115:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177533 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762936.968125:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3edba8/0xdf5a7ac59f177533 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762936.968217:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067531520:12345-192.168.203.37@tcp:4:dd.0 Request processed in 21971us (43092us total) trans 4294968103 rc 0/0 00000100:00100000:2.0:1789762936.968229:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 995 00000100:00100000:2.0:1789762936.968248:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410193204328 time 410193290295 state 0x2 ref 0 00000100:00100000:1.0:1789762936.972325:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762936.972391:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067531648 00000100:00100000:2.0:1789762936.972403:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067531648 00000100:00100000:2.0:1789762936.972473:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410193204328 time 410193290295 state 0x0 ref 1 00000100:00100000:2.0:1789762936.972500:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410243290295 time 410193290295 state 0x1 ref 1 00000100:00100000:3.0:1789762936.996404:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410243290295 time 410243460303 state 0x0 ref 1 00000100:00100000:3.0:1789762936.996446:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067531648:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762936.996519:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 75497472, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x24009 00010000:00010000:3.0:1789762936.996532:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17753a lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.996554:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762936.996561:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762936.996570:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17753a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762936.996590:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17753a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762937.018401:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968104 is committed 00002000:00010000:2.0:1789762937.018510:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 75497472 -> 76546048 00002000:00010000:2.0:1789762937.018519:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 147465 -> 149513 00010000:00010000:2.0:1789762937.018534:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17753a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.018551:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17753a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.018562:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17753a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.018572:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762937.018578:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17753a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.018590:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17753a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.018614:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17753a lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.018628:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080a412f0/0xdf5a7ac59f17753a lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762937.018755:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067531648:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22323us (46431us total) trans 4294968104 rc 0/0 00000100:00100000:2.0:1789762937.018772:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 996 00000100:00100000:2.0:1789762937.018788:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410243290295 time 410243460303 state 0x2 ref 0 00000100:00100000:1.0:1789762937.023179:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762937.023249:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067531776 00000100:00100000:2.0:1789762937.023266:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067531776 00000100:00080000:1.0:1789762937.023344:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762937.023386:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410243290295 time 410243460303 state 0x0 ref 1 00000100:00100000:2.0:1789762937.023415:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410293460303 time 410243460303 state 0x1 ref 1 00000100:00100000:2.0:1789762937.046506:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410293460303 time 410293572240 state 0x0 ref 1 00000100:00100000:2.0:1789762937.046532:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067531776:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762937.046583:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 76546048, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x24809 00010000:00010000:2.0:1789762937.046593:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f177541 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.046613:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:2.0:1789762937.046620:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762937.046632:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f177541 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.046656:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f177541 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762937.068566:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968105 is committed 00002000:00010000:2.0:1789762937.068679:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 76546048 -> 77594624 00002000:00010000:2.0:1789762937.068692:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 149513 -> 151561 00010000:00010000:2.0:1789762937.068710:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f177541 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.068731:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f177541 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.068745:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f177541 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.068760:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762937.068768:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f177541 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.068787:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f177541 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.068819:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f177541 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.068857:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fec7b57/0xdf5a7ac59f177541 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762937.069022:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067531776:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22492us (45841us total) trans 4294968105 rc 0/0 00000100:00100000:2.0:1789762937.069039:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 997 00000100:00100000:2.0:1789762937.069054:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410293460303 time 410293572240 state 0x2 ref 0 00000100:00100000:1.0:1789762937.073206:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762937.073269:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067531904 00000100:00100000:2.0:1789762937.073282:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067531904 00000100:00100000:2.0:1789762937.073355:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410293460303 time 410293572240 state 0x0 ref 1 00000100:00100000:2.0:1789762937.073373:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410343572240 time 410293572240 state 0x1 ref 1 00000100:00100000:3.0:1789762937.096639:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410343572240 time 410343700111 state 0x0 ref 1 00000100:00100000:3.0:1789762937.096674:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067531904:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762937.096736:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 77594624, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x25009 00010000:00010000:3.0:1789762937.096748:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f177548 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.096768:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762937.096774:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762937.096782:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f177548 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.096800:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f177548 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762937.119348:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968106 is committed 00002000:00010000:2.0:1789762937.119492:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:2.0:1789762937.119503:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 151561 -> 153609 00010000:00010000:2.0:1789762937.119525:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f177548 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.119549:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f177548 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.119568:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f177548 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.119581:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762937.119588:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f177548 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.119599:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f177548 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.119632:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f177548 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.119642:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001628ad8c/0xdf5a7ac59f177548 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762937.119758:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067531904:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23091us (46548us total) trans 4294968106 rc 0/0 00000100:00100000:2.0:1789762937.119777:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 998 00000100:00100000:2.0:1789762937.119797:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410343572240 time 410343700111 state 0x2 ref 0 00000100:00100000:1.0:1789762937.124029:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762937.124098:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067532032 00000100:00100000:2.0:1789762937.124114:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067532032 00000100:00100000:2.0:1789762937.124184:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410343572240 time 410343700111 state 0x0 ref 1 00000100:00100000:2.0:1789762937.124203:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410393700111 time 410343700111 state 0x1 ref 1 00000100:00100000:3.0:1789762937.146753:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410393700111 time 410393817669 state 0x0 ref 1 00000100:00100000:3.0:1789762937.146781:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067532032:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762937.146865:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 78643200, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x25809 00010000:00010000:3.0:1789762937.146877:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f17754f lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.146897:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762937.146903:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762937.146910:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f17754f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.146955:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f17754f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762937.168458:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968107 is committed 00002000:00010000:1.0:1789762937.168540:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:1.0:1789762937.168553:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 153609 -> 155657 00010000:00010000:1.0:1789762937.168572:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f17754f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.168596:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f17754f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.168608:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f17754f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.168618:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.168624:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f17754f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.168643:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f17754f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.168666:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f17754f lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.168675:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d2716b6/0xdf5a7ac59f17754f lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.168789:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067532032:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22011us (44765us total) trans 4294968107 rc 0/0 00000100:00100000:1.0:1789762937.168810:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 999 00000100:00100000:1.0:1789762937.168858:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410393700111 time 410393817669 state 0x2 ref 0 00000100:00100000:1.0:1789762937.172780:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762937.172902:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067532160 00000100:00100000:2.0:1789762937.172913:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067532160 00000100:00100000:2.0:1789762937.173025:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410393700111 time 410393817669 state 0x0 ref 1 00000100:00100000:2.0:1789762937.173046:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410443817669 time 410393817669 state 0x1 ref 1 00000100:00100000:3.0:1789762937.197027:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410443817669 time 410444084740 state 0x0 ref 1 00000100:00100000:3.0:1789762937.197094:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9627100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067532160:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762937.197176:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 79691776, mtime: 0x6aad9d78, atime: 0x0, ctime: 0x6aad9d78, blocks: 0x26009 00010000:00010000:3.0:1789762937.197190:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177556 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.197211:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762937.197217:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762937.197225:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177556 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.197244:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177556 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762937.219419:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968108 is committed 00002000:00010000:1.0:1789762937.219467:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 79691776 -> 80740352 00002000:00010000:1.0:1789762937.219477:0:8058:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb mtime from disk: 1789762936 -> 1789762937 00002000:00010000:1.0:1789762937.219483:0:8058:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb ctime from disk: 1789762936 -> 1789762937 00002000:00010000:1.0:1789762937.219488:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 155657 -> 157705 00010000:00010000:1.0:1789762937.219510:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177556 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.219537:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177556 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.219571:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177556 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.219588:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.219598:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177556 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.219617:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177556 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.219647:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177556 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.219657:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f177556 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.219756:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9627100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067532160:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22683us (46976us total) trans 4294968108 rc 0/0 00000100:00100000:1.0:1789762937.219770:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1000 00000100:00100000:1.0:1789762937.219786:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410443817669 time 410444084740 state 0x2 ref 0 00000100:00100000:0.0:1789762937.224087:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762937.224170:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067532288 00000100:00100000:2.0:1789762937.224183:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067532288 00000100:00100000:2.0:1789762937.224245:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410443817669 time 410444084740 state 0x0 ref 1 00000100:00100000:2.0:1789762937.224264:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410494084740 time 410444084740 state 0x1 ref 1 00000100:00100000:3.0:1789762937.247167:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410494084740 time 410494232144 state 0x0 ref 1 00000100:00100000:3.0:1789762937.247193:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067532288:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762937.247243:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 80740352, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x26809 00010000:00010000:3.0:1789762937.247253:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17755d lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.247272:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762937.247277:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762937.247285:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17755d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.247301:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17755d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762937.270004:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968109 is committed 00002000:00010000:1.0:1789762937.270132:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 80740352 -> 81788928 00002000:00010000:1.0:1789762937.270141:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 157705 -> 159753 00010000:00010000:1.0:1789762937.270155:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17755d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.270181:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17755d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.270192:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17755d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.270201:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.270207:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17755d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.270217:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17755d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.270239:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17755d lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.270249:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f17755d lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.270362:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067532288:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23170us (46271us total) trans 4294968109 rc 0/0 00000100:00100000:1.0:1789762937.270382:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1001 00000100:00100000:1.0:1789762937.270404:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410494084740 time 410494232144 state 0x2 ref 0 00000100:00100000:2.0:1789762937.274466:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762937.274537:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067532416 00000100:00100000:1.0:1789762937.274549:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067532416 00000100:00100000:1.0:1789762937.274608:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410494084740 time 410494232144 state 0x0 ref 1 00000100:00100000:1.0:1789762937.274626:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410544232144 time 410494232144 state 0x1 ref 1 00000100:00100000:3.0:1789762937.297306:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410544232144 time 410544369558 state 0x0 ref 1 00000100:00100000:3.0:1789762937.297338:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eadc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067532416:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762937.297395:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 81788928, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x27009 00010000:00010000:3.0:1789762937.297406:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f177564 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.297426:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762937.297432:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762937.297440:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f177564 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.297457:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f177564 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762937.320540:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968110 is committed 00002000:00010000:1.0:1789762937.320661:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 81788928 -> 82837504 00002000:00010000:1.0:1789762937.320671:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 159753 -> 161801 00010000:00010000:1.0:1789762937.320688:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f177564 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.320707:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f177564 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.320719:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f177564 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.320728:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.320734:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f177564 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.320747:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f177564 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.320772:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f177564 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.320789:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aae2a713/0xdf5a7ac59f177564 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.321029:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eadc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067532416:12345-192.168.203.37@tcp:4:dd.0 Request processed in 23690us (46555us total) trans 4294968110 rc 0/0 00000100:00100000:1.0:1789762937.321054:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1002 00000100:00100000:1.0:1789762937.321079:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410544232144 time 410544369558 state 0x2 ref 0 00000100:00100000:0.0:1789762937.325561:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762937.325661:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067532544 00000100:00100000:1.0:1789762937.325678:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067532544 00000100:00100000:1.0:1789762937.325752:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410544232144 time 410544369558 state 0x0 ref 1 00000100:00100000:1.0:1789762937.325776:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410594369558 time 410544369558 state 0x1 ref 1 00000100:00100000:3.0:1789762937.347522:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410594369558 time 410594587952 state 0x0 ref 1 00000100:00100000:3.0:1789762937.347551:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067532544:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762937.347607:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 82837504, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x27809 00010000:00010000:3.0:1789762937.347618:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17756b lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.347638:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762937.347645:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762937.347653:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17756b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.347671:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17756b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762937.370130:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 82837504 -> 83886080 00000001:00080000:0.0:1789762937.370134:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968111 is committed 00002000:00010000:3.0:1789762937.370141:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 161801 -> 163849 00010000:00010000:3.0:1789762937.370160:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17756b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.370186:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17756b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.370198:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17756b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.370207:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762937.370213:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17756b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.370224:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17756b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.370246:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17756b lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.370256:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17756b lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762937.370353:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067532544:12345-192.168.203.37@tcp:4:dd.0 Request processed in 22808us (44793us total) trans 4294968111 rc 0/0 00000100:00100000:3.0:1789762937.370365:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1003 00000100:00100000:3.0:1789762937.370380:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410594369558 time 410594587952 state 0x2 ref 0 00000100:00100000:1.0:1789762937.374503:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762937.374570:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067532672 00000100:00100000:3.0:1789762937.374583:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067532672 00000100:00100000:3.0:1789762937.374638:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410594369558 time 410594587952 state 0x0 ref 1 00000100:00100000:3.0:1789762937.374656:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410644587952 time 410594587952 state 0x1 ref 1 00000100:00100000:2.0:1789762937.397674:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410644587952 time 410644737420 state 0x0 ref 1 00000100:00100000:2.0:1789762937.397703:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9624380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067532672:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762937.397759:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 83886080, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x28009 00010000:00010000:2.0:1789762937.397770:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f177572 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.397792:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:2.0:1789762937.397800:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762937.397812:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f177572 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.397858:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f177572 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762937.418974:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968112 is committed 00002000:00010000:1.0:1789762937.419103:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 83886080 -> 84934656 00002000:00010000:1.0:1789762937.419114:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 163849 -> 165897 00010000:00010000:1.0:1789762937.419132:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f177572 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.419150:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f177572 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.419162:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f177572 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.419171:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.419178:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f177572 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.419191:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f177572 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.419216:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f177572 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.419226:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002040ccb8/0xdf5a7ac59f177572 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.419337:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9624380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067532672:12345-192.168.203.37@tcp:4:dd.0 Request processed in 21638us (44833us total) trans 4294968112 rc 0/0 00000100:00100000:1.0:1789762937.419352:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1004 00000100:00100000:1.0:1789762937.419368:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410644587952 time 410644737420 state 0x2 ref 0 00000100:00100000:1.0:1789762937.424028:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762937.424117:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067532800 00000100:00100000:3.0:1789762937.424132:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067532800 00000100:00100000:3.0:1789762937.424196:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410644587952 time 410644737420 state 0x0 ref 1 00000100:00100000:3.0:1789762937.424216:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410694737420 time 410644737420 state 0x1 ref 1 00000100:00100000:2.0:1789762937.447804:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410694737420 time 410694868451 state 0x0 ref 1 00000100:00100000:2.0:1789762937.447870:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9626a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067532800:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762937.447966:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 84934656, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x28809 00010000:00010000:2.0:1789762937.447979:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177579 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.448000:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:2.0:1789762937.448006:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762937.448014:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177579 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.448032:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177579 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762937.469167:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968113 is committed 00002000:00010000:1.0:1789762937.469249:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 84934656 -> 85983232 00002000:00010000:1.0:1789762937.469257:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 165897 -> 167945 00010000:00010000:1.0:1789762937.469271:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177579 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.469286:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177579 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.469297:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177579 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.469314:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.469320:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177579 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.469330:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177579 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.469350:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177579 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.469359:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000957f5957/0xdf5a7ac59f177579 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.469454:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9626a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067532800:12345-192.168.203.37@tcp:4:dd.0 Request processed in 21594us (45428us total) trans 4294968113 rc 0/0 00000100:00100000:1.0:1789762937.469472:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1005 00000100:00100000:1.0:1789762937.469491:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410694737420 time 410694868451 state 0x2 ref 0 00010000:00010000:0.0:1789762937.472196:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762937.473110:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762937.473169:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067532928 00000100:00100000:3.0:1789762937.473181:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067532928 00000100:00100000:3.0:1789762937.473232:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410694737420 time 410694868451 state 0x0 ref 1 00000100:00100000:3.0:1789762937.473248:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410744868451 time 410694868451 state 0x1 ref 1 00000100:00100000:1.0:1789762937.497897:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410744868451 time 410744967665 state 0x0 ref 1 00000100:00100000:1.0:1789762937.497940:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9624e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067532928:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762937.497981:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 85983232, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x29009 00010000:00010000:1.0:1789762937.497989:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177580 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.498004:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:1.0:1789762937.498007:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762937.498012:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177580 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.498023:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177580 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762937.509461:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968114 is committed 00002000:00010000:1.0:1789762937.509492:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 85983232 -> 87031808 00002000:00010000:1.0:1789762937.509498:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 167945 -> 169993 00010000:00010000:1.0:1789762937.509508:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177580 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.509517:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177580 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.509523:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177580 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.509528:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.509532:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177580 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.509538:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177580 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.509556:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177580 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.509561:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177580 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.509628:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9624e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067532928:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11716us (36521us total) trans 4294968114 rc 0/0 00000100:00100000:1.0:1789762937.509636:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1006 00000100:00100000:1.0:1789762937.509645:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410744868451 time 410744967665 state 0x2 ref 0 00000100:00100000:0.0:1789762937.512247:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762937.512281:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067533056 00000100:00100000:1.0:1789762937.512290:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067533056 00000100:00100000:1.0:1789762937.512338:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410744868451 time 410744967665 state 0x0 ref 1 00000100:00100000:1.0:1789762937.512349:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410794967665 time 410744967665 state 0x1 ref 1 00000100:00100000:1.0:1789762937.547949:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410794967665 time 410795021734 state 0x0 ref 1 00000100:00100000:1.0:1789762937.547965:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067533056:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762937.548000:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 87031808, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x29809 00010000:00010000:1.0:1789762937.548006:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177587 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.548022:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:1.0:1789762937.548026:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762937.548030:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177587 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.548040:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177587 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762937.559549:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968115 is committed 00002000:00010000:1.0:1789762937.559572:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 87031808 -> 88080384 00002000:00010000:1.0:1789762937.559577:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 169993 -> 172041 00010000:00010000:1.0:1789762937.559589:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177587 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.559599:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177587 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.559607:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177587 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.559612:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.559616:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177587 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.559622:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177587 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.559641:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177587 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.559647:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f177587 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.559716:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067533056:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11752us (47467us total) trans 4294968115 rc 0/0 00000100:00100000:1.0:1789762937.559724:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1007 00000100:00100000:1.0:1789762937.559734:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410794967665 time 410795021734 state 0x2 ref 0 00000100:00100000:0.0:1789762937.562234:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762937.562303:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067533184 00000100:00100000:1.0:1789762937.562311:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067533184 00000100:00100000:1.0:1789762937.562350:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410794967665 time 410795021734 state 0x0 ref 1 00000100:00100000:1.0:1789762937.562360:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410845021734 time 410795021734 state 0x1 ref 1 00000100:00100000:1.0:1789762937.598057:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410845021734 time 410845117480 state 0x0 ref 1 00000100:00100000:1.0:1789762937.598092:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067533184:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762937.598155:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 88080384, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x2a009 00010000:00010000:1.0:1789762937.598174:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17758e lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.598194:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:1.0:1789762937.598198:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762937.598203:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17758e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.598214:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17758e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762937.610248:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968116 is committed 00002000:00010000:1.0:1789762937.610307:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 88080384 -> 89128960 00002000:00010000:1.0:1789762937.610323:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 172041 -> 174089 00010000:00010000:1.0:1789762937.610335:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17758e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.610349:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17758e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.610356:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17758e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.610361:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.610365:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17758e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.610371:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17758e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.610401:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17758e lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.610406:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f17758e lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.610476:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067533184:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12396us (48238us total) trans 4294968116 rc 0/0 00000100:00100000:1.0:1789762937.610487:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1008 00000100:00100000:1.0:1789762937.610496:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410845021734 time 410845117480 state 0x2 ref 0 00000100:00100000:0.0:1789762937.613213:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762937.613248:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067533312 00000100:00100000:1.0:1789762937.613255:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067533312 00000100:00100000:1.0:1789762937.613297:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410845021734 time 410845117480 state 0x0 ref 1 00000100:00100000:1.0:1789762937.613325:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410895117480 time 410845117480 state 0x1 ref 1 00000100:00100000:3.0:1789762937.648120:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410895117480 time 410895188598 state 0x0 ref 1 00000100:00100000:3.0:1789762937.648145:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067533312:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762937.648184:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 89128960, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x2a809 00010000:00010000:3.0:1789762937.648191:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f177595 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.648207:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762937.648210:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762937.648215:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f177595 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.648225:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f177595 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762937.661051:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968117 is committed 00002000:00010000:3.0:1789762937.661065:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 89128960 -> 90177536 00002000:00010000:3.0:1789762937.661072:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 174089 -> 176137 00010000:00010000:3.0:1789762937.661085:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f177595 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.661099:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f177595 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.661106:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f177595 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.661112:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762937.661115:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f177595 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.661122:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f177595 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.661141:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f177595 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.661147:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f177595 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762937.661230:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067533312:12345-192.168.203.37@tcp:4:dd.0 Request processed in 13091us (48015us total) trans 4294968117 rc 0/0 00000100:00100000:3.0:1789762937.661241:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1009 00000100:00100000:3.0:1789762937.661251:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410895117480 time 410895188598 state 0x2 ref 0 00000100:00100000:0.0:1789762937.664044:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762937.664101:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067533440 00000100:00100000:1.0:1789762937.664109:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067533440 00000100:00100000:1.0:1789762937.664160:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410895117480 time 410895188598 state 0x0 ref 1 00000100:00100000:1.0:1789762937.664171:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410945188598 time 410895188598 state 0x1 ref 1 00000100:00100000:3.0:1789762937.698218:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410945188598 time 410945285110 state 0x0 ref 1 00000100:00100000:3.0:1789762937.698244:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067533440:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762937.698312:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 90177536, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x2b009 00010000:00010000:3.0:1789762937.698322:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17759c lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.698339:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:3.0:1789762937.698344:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762937.698350:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17759c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.698362:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17759c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762937.711074:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968118 is committed 00002000:00010000:3.0:1789762937.711077:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 90177536 -> 91226112 00002000:00010000:3.0:1789762937.711083:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 176137 -> 178185 00010000:00010000:3.0:1789762937.711093:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17759c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.711103:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17759c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.711109:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17759c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.711115:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762937.711120:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17759c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.711130:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17759c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.711150:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17759c lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.711159:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f17759c lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762937.711242:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067533440:12345-192.168.203.37@tcp:4:dd.0 Request processed in 13007us (47196us total) trans 4294968118 rc 0/0 00000100:00100000:3.0:1789762937.711251:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1010 00000100:00100000:3.0:1789762937.711260:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410945188598 time 410945285110 state 0x2 ref 0 00000100:00100000:0.0:1789762937.713651:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762937.713735:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067533568 00000100:00100000:1.0:1789762937.713743:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067533568 00000100:00100000:1.0:1789762937.713793:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410945188598 time 410945285110 state 0x0 ref 1 00000100:00100000:1.0:1789762937.713805:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410995285110 time 410945285110 state 0x1 ref 1 00000100:00100000:2.0:1789762937.748302:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410995285110 time 410995372434 state 0x0 ref 1 00000100:00100000:2.0:1789762937.748319:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067533568:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762937.748377:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 91226112, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x2b809 00010000:00010000:2.0:1789762937.748383:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1775a3 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.748394:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=410s v=(0 0) 00010000:00010000:2.0:1789762937.748398:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762937.748403:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1775a3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.748414:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1775a3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762937.760329:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968119 is committed 00002000:00010000:1.0:1789762937.760355:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 91226112 -> 92274688 00002000:00010000:1.0:1789762937.760361:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 178185 -> 180233 00010000:00010000:1.0:1789762937.760372:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1775a3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.760385:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1775a3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.760395:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1775a3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.760404:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.760408:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1775a3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.760419:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1775a3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.760439:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1775a3 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.760447:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1775a3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.760512:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067533568:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12193us (46858us total) trans 4294968119 rc 0/0 00000100:00100000:1.0:1789762937.760522:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1011 00000100:00100000:1.0:1789762937.760533:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410995285110 time 410995372434 state 0x2 ref 0 00000100:00100000:0.0:1789762937.762874:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762937.762931:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067533696 00000100:00100000:3.0:1789762937.762939:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067533696 00000100:00100000:3.0:1789762937.762976:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 410995285110 time 410995372434 state 0x0 ref 1 00000100:00100000:3.0:1789762937.762987:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411045372434 time 410995372434 state 0x1 ref 1 00000100:00100000:2.0:1789762937.798402:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411045372434 time 411045469822 state 0x0 ref 1 00000100:00100000:2.0:1789762937.798424:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067533696:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762937.798465:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 92274688, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x2c009 00010000:00010000:2.0:1789762937.798472:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f1775aa lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.798485:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:2.0:1789762937.798489:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762937.798493:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f1775aa lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762937.798505:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f1775aa lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762937.810389:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968120 is committed 00002000:00010000:1.0:1789762937.810444:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 92274688 -> 93323264 00002000:00010000:1.0:1789762937.810450:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 180233 -> 182281 00010000:00010000:1.0:1789762937.810471:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f1775aa lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.810484:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f1775aa lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.810496:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f1775aa lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.810504:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.810510:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f1775aa lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.810521:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f1775aa lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.810538:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f1775aa lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.810564:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f1775aa lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.810630:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067533696:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12211us (47753us total) trans 4294968120 rc 0/0 00000100:00100000:1.0:1789762937.810640:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1012 00000100:00100000:1.0:1789762937.810649:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411045372434 time 411045469822 state 0x2 ref 0 00000100:00100000:1.0:1789762937.812997:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762937.813037:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067533824 00000100:00100000:3.0:1789762937.813046:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067533824 00000100:00100000:3.0:1789762937.813082:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411045372434 time 411045469822 state 0x0 ref 1 00000100:00100000:3.0:1789762937.813092:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411095469822 time 411045469822 state 0x1 ref 1 00000100:00100000:3.0:1789762937.848452:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411095469822 time 411095522908 state 0x0 ref 1 00000100:00100000:3.0:1789762937.848470:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9625500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067533824:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762937.848506:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 93323264, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x2c809 00010000:00010000:3.0:1789762937.848512:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1775b1 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.848524:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:3.0:1789762937.848527:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762937.848532:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1775b1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.848543:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1775b1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762937.859961:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968121 is committed 00002000:00010000:1.0:1789762937.859964:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 93323264 -> 94371840 00002000:00010000:1.0:1789762937.859969:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 182281 -> 184329 00010000:00010000:1.0:1789762937.859980:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1775b1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.859993:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1775b1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.860004:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1775b1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.860013:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.860018:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1775b1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.860029:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1775b1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.860049:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1775b1 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.860059:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4cbda34/0xdf5a7ac59f1775b1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.860123:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9625500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067533824:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11655us (47124us total) trans 4294968121 rc 0/0 00000100:00100000:1.0:1789762937.860133:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1013 00000100:00100000:1.0:1789762937.860143:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411095469822 time 411095522908 state 0x2 ref 0 00000100:00100000:0.0:1789762937.862207:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762937.862275:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067533952 00000100:00100000:1.0:1789762937.862283:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067533952 00000100:00100000:1.0:1789762937.862319:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411095469822 time 411095522908 state 0x0 ref 1 00000100:00100000:1.0:1789762937.862331:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411145522908 time 411095522908 state 0x1 ref 1 00000100:00100000:1.0:1789762937.898519:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411145522908 time 411145588128 state 0x0 ref 1 00000100:00100000:1.0:1789762937.898539:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067533952:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762937.898580:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 94371840, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x2d009 00010000:00010000:1.0:1789762937.898587:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775b8 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.898604:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:1.0:1789762937.898607:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762937.898612:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775b8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.898623:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775b8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762937.910870:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968122 is committed 00002000:00010000:1.0:1789762937.910895:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 94371840 -> 95420416 00002000:00010000:1.0:1789762937.910902:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 184329 -> 186377 00010000:00010000:1.0:1789762937.910913:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775b8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.910945:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775b8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.910953:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775b8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.910959:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.910962:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775b8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.910969:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775b8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.910987:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775b8 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.910993:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775b8 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.911055:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067533952:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12521us (48846us total) trans 4294968122 rc 0/0 00000100:00100000:1.0:1789762937.911063:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1014 00000100:00100000:1.0:1789762937.911071:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411145522908 time 411145588128 state 0x2 ref 0 00000100:00100000:2.0:1789762937.913468:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762937.913512:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067534080 00000100:00100000:1.0:1789762937.913520:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067534080 00000100:00100000:1.0:1789762937.913562:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411145522908 time 411145588128 state 0x0 ref 1 00000100:00100000:1.0:1789762937.913573:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411195588128 time 411145588128 state 0x1 ref 1 00000100:00100000:1.0:1789762937.948589:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411195588128 time 411195655323 state 0x0 ref 1 00000100:00100000:1.0:1789762937.948614:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067534080:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762937.948665:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 95420416, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x2d809 00010000:00010000:1.0:1789762937.948671:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775bf lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.948689:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:1.0:1789762937.948693:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762937.948697:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775bf lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.948707:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775bf lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762937.960109:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968123 is committed 00002000:00010000:1.0:1789762937.960133:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 95420416 -> 96468992 00002000:00010000:1.0:1789762937.960140:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 186377 -> 188427 00010000:00010000:1.0:1789762937.960152:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775bf lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.960164:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775bf lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.960174:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775bf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.960179:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762937.960183:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775bf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.960189:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775bf lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.960208:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775bf lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.960214:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eeb7fe6f/0xdf5a7ac59f1775bf lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.960272:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067534080:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11668us (46804us total) trans 4294968123 rc 0/0 00000100:00100000:1.0:1789762937.960280:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1015 00000100:00100000:1.0:1789762937.960288:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411195588128 time 411195655323 state 0x2 ref 0 00000100:00100000:0.0:1789762937.962565:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762937.962600:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067534208 00000100:00100000:1.0:1789762937.962608:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067534208 00000100:00100000:1.0:1789762937.962646:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411195588128 time 411195655323 state 0x0 ref 1 00000100:00100000:1.0:1789762937.962657:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411245655323 time 411195655323 state 0x1 ref 1 00000100:00100000:3.0:1789762937.998695:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411245655323 time 411245763450 state 0x0 ref 1 00000100:00100000:3.0:1789762937.998716:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067534208:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762937.998762:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 96468992, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x2e00b 00010000:00010000:3.0:1789762937.998769:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1775c6 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.998783:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:3.0:1789762937.998787:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762937.998792:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1775c6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762937.998803:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1775c6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762938.010774:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968124 is committed 00002000:00010000:1.0:1789762938.010838:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 96468992 -> 97517568 00002000:00010000:1.0:1789762938.010843:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 188427 -> 190475 00010000:00010000:1.0:1789762938.010852:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1775c6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.010862:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1775c6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.010873:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1775c6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.010878:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762938.010881:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1775c6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.010887:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1775c6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.010901:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1775c6 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.010906:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbfc7ef0/0xdf5a7ac59f1775c6 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762938.010986:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067534208:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12275us (48422us total) trans 4294968124 rc 0/0 00000100:00100000:1.0:1789762938.010994:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1016 00000100:00100000:1.0:1789762938.011002:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411245655323 time 411245763450 state 0x2 ref 0 00000100:00100000:2.0:1789762938.013192:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762938.013261:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067534336 00000100:00100000:1.0:1789762938.013269:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067534336 00000100:00100000:1.0:1789762938.013306:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411245655323 time 411245763450 state 0x0 ref 1 00000100:00100000:1.0:1789762938.013319:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411295763450 time 411245763450 state 0x1 ref 1 00000100:00100000:3.0:1789762938.048792:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411295763450 time 411295860333 state 0x0 ref 1 00000100:00100000:3.0:1789762938.048831:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067534336:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762938.048879:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 97517568, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x2e80b 00010000:00010000:3.0:1789762938.048887:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f1775cd lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.048900:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:3.0:1789762938.048903:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762938.048908:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f1775cd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.048940:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f1775cd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762938.061178:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968125 is committed 00002000:00010000:1.0:1789762938.061209:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 97517568 -> 98566144 00002000:00010000:1.0:1789762938.061215:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 190475 -> 192523 00010000:00010000:1.0:1789762938.061227:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f1775cd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.061238:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f1775cd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.061248:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f1775cd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.061256:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762938.061261:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f1775cd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.061271:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f1775cd lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.061291:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f1775cd lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.061300:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b259d529/0xdf5a7ac59f1775cd lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762938.061380:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067534336:12345-192.168.203.37@tcp:4:dd.0 Request processed in 12568us (48185us total) trans 4294968125 rc 0/0 00000100:00100000:1.0:1789762938.061391:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1017 00000100:00100000:1.0:1789762938.061403:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411295763450 time 411295860333 state 0x2 ref 0 00000100:00100000:0.0:1789762938.063769:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762938.063813:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067534464 00000100:00100000:1.0:1789762938.063833:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067534464 00000100:00100000:1.0:1789762938.063877:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411295763450 time 411295860333 state 0x0 ref 1 00000100:00100000:1.0:1789762938.063892:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411345860333 time 411295860333 state 0x1 ref 1 00000100:00100000:2.0:1789762938.098964:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411345860333 time 411345993614 state 0x0 ref 1 00000100:00100000:2.0:1789762938.099004:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067534464:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:2.0:1789762938.099059:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 98566144, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x2f00b 00010000:00010000:2.0:1789762938.099066:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1775d4 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.099079:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:2.0:1789762938.099083:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762938.099088:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1775d4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.099099:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1775d4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762938.113103:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968126 is committed 00002000:00010000:1.0:1789762938.113151:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 98566144 -> 99614720 00002000:00010000:1.0:1789762938.113157:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 192523 -> 194571 00010000:00010000:1.0:1789762938.113172:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1775d4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.113187:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1775d4 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.113198:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1775d4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.113208:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762938.113214:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1775d4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.113231:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1775d4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.113255:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1775d4 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.113265:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a558e1d5/0xdf5a7ac59f1775d4 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762938.113365:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067534464:12345-192.168.203.37@tcp:4:dd.0 Request processed in 14376us (49591us total) trans 4294968126 rc 0/0 00000100:00100000:1.0:1789762938.113379:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1018 00000100:00100000:1.0:1789762938.113393:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411345860333 time 411345993614 state 0x2 ref 0 00000100:00100000:0.0:1789762938.116073:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762938.116121:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067534592 00000100:00100000:1.0:1789762938.116131:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067534592 00000100:00100000:1.0:1789762938.116179:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411345860333 time 411345993614 state 0x0 ref 1 00000100:00100000:1.0:1789762938.116195:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411395993614 time 411345993614 state 0x1 ref 1 00000100:00100000:3.0:1789762938.149039:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411395993614 time 411396105368 state 0x0 ref 1 00000100:00100000:3.0:1789762938.149071:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067534592:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762938.149124:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 99614720, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x2f80b 00010000:00010000:3.0:1789762938.149131:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775db lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.149145:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:3.0:1789762938.149149:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762938.149154:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775db lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.149166:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775db lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762938.165260:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 99614720 -> 100663296 00002000:00010000:3.0:1789762938.165269:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 194571 -> 196619 00010000:00010000:3.0:1789762938.165282:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775db lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762938.165288:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968127 is committed 00010000:00010000:3.0:1789762938.165307:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775db lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.165321:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775db lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.165327:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762938.165332:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775db lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.165339:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775db lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.165354:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775db lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.165360:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775db lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762938.165428:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067534592:12345-192.168.203.37@tcp:4:dd.0 Request processed in 16371us (49354us total) trans 4294968127 rc 0/0 00000100:00100000:3.0:1789762938.165436:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1019 00000100:00100000:3.0:1789762938.165445:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411395993614 time 411396105368 state 0x2 ref 0 00000100:00100000:2.0:1789762938.167945:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762938.168004:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067534720 00000100:00100000:1.0:1789762938.168014:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067534720 00000100:00100000:1.0:1789762938.168062:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411395993614 time 411396105368 state 0x0 ref 1 00000100:00100000:1.0:1789762938.168079:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411446105368 time 411396105368 state 0x1 ref 1 00000100:00100000:1.0:1789762938.199118:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411446105368 time 411446183114 state 0x0 ref 1 00000100:00100000:1.0:1789762938.199146:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067534720:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:1.0:1789762938.199197:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 100663296, mtime: 0x6aad9d79, atime: 0x0, ctime: 0x6aad9d79, blocks: 0x3000b 00010000:00010000:1.0:1789762938.199204:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1775e2 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.199218:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:1.0:1789762938.199222:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762938.199226:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1775e2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.199238:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1775e2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762938.213400:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968128 is committed 00002000:00010000:1.0:1789762938.213451:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 100663296 -> 101711872 00002000:00010000:1.0:1789762938.213458:0:13300:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb mtime from disk: 1789762937 -> 1789762938 00002000:00010000:1.0:1789762938.213461:0:13300:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb ctime from disk: 1789762937 -> 1789762938 00002000:00010000:1.0:1789762938.213463:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 196619 -> 198667 00010000:00010000:1.0:1789762938.213477:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1775e2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.213491:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1775e2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.213507:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1775e2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.213516:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762938.213521:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1775e2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.213532:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1775e2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.213554:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1775e2 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.213564:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1775e2 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762938.213659:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067534720:12345-192.168.203.37@tcp:4:dd.0 Request processed in 14521us (45711us total) trans 4294968128 rc 0/0 00000100:00100000:1.0:1789762938.213672:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1020 00000100:00100000:1.0:1789762938.213685:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411446105368 time 411446183114 state 0x2 ref 0 00000100:00100000:2.0:1789762938.216214:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762938.216253:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067534848 00000100:00100000:1.0:1789762938.216262:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067534848 00000100:00100000:1.0:1789762938.216303:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411446105368 time 411446183114 state 0x0 ref 1 00000100:00100000:1.0:1789762938.216318:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411496183114 time 411446183114 state 0x1 ref 1 00000100:00100000:3.0:1789762938.249191:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411496183114 time 411496263123 state 0x0 ref 1 00000100:00100000:3.0:1789762938.249209:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067534848:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762938.249249:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 101711872, mtime: 0x6aad9d7a, atime: 0x0, ctime: 0x6aad9d7a, blocks: 0x3080b 00010000:00010000:3.0:1789762938.249258:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775e9 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.249274:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:3.0:1789762938.249280:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762938.249287:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775e9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.249302:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775e9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762938.260679:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968129 is committed 00002000:00010000:3.0:1789762938.260679:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 101711872 -> 102760448 00002000:00010000:3.0:1789762938.260686:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 198667 -> 200715 00010000:00010000:3.0:1789762938.260696:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775e9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.260711:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775e9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.260717:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775e9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.260722:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762938.260726:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775e9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.260732:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775e9 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.260745:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775e9 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.260751:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775e9 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762938.260805:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067534848:12345-192.168.203.37@tcp:4:dd.0 Request processed in 11600us (44592us total) trans 4294968129 rc 0/0 00000100:00100000:3.0:1789762938.260812:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1021 00000100:00100000:3.0:1789762938.260832:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411496183114 time 411496263123 state 0x2 ref 0 00000100:00100000:0.0:1789762938.263079:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762938.263120:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067534976 00000100:00100000:3.0:1789762938.263127:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067534976 00000100:00100000:3.0:1789762938.263162:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411496183114 time 411496263123 state 0x0 ref 1 00000100:00100000:3.0:1789762938.263173:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411546263123 time 411496263123 state 0x1 ref 1 00000100:00100000:3.0:1789762938.299249:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411546263123 time 411546313039 state 0x0 ref 1 00000100:00100000:3.0:1789762938.299275:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067534976:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762938.299323:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 102760448, mtime: 0x6aad9d7a, atime: 0x0, ctime: 0x6aad9d7a, blocks: 0x3100b 00010000:00010000:3.0:1789762938.299333:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f0 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.299348:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:3.0:1789762938.299352:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762938.299357:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.299367:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762938.312146:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968130 is committed 00002000:00010000:3.0:1789762938.312152:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 102760448 -> 103809024 00002000:00010000:3.0:1789762938.312157:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 200715 -> 202763 00010000:00010000:3.0:1789762938.312168:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.312177:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.312185:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.312191:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762938.312194:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.312200:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.312214:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f0 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.312219:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f0 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762938.312280:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067534976:12345-192.168.203.37@tcp:4:dd.0 Request processed in 13012us (49199us total) trans 4294968130 rc 0/0 00000100:00100000:3.0:1789762938.312287:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1022 00000100:00100000:3.0:1789762938.312296:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411546263123 time 411546313039 state 0x2 ref 0 00000100:00100000:2.0:1789762938.314549:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762938.314600:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067535104 00000100:00100000:1.0:1789762938.314610:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067535104 00000100:00100000:1.0:1789762938.314648:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411546263123 time 411546313039 state 0x0 ref 1 00000100:00100000:1.0:1789762938.314659:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411596313039 time 411546313039 state 0x1 ref 1 00000100:00100000:3.0:1789762938.349320:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411596313039 time 411596389560 state 0x0 ref 1 00000100:00100000:3.0:1789762938.349345:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067535104:12345-192.168.203.37@tcp:4:dd.0 00002000:00010000:3.0:1789762938.349389:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 103809024, mtime: 0x6aad9d7a, atime: 0x0, ctime: 0x6aad9d7a, blocks: 0x3180b 00010000:00010000:3.0:1789762938.349396:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f7 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.349408:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:3.0:1789762938.349412:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762938.349417:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.349427:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762938.363849:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 103809024 -> 104857600 00002000:00010000:1.0:1789762938.363855:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb blocks from disk: 202763 -> 204811 00010000:00010000:1.0:1789762938.363865:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.363875:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.363882:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.363887:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762938.363890:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.363899:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762938.363914:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968131 is committed 00010000:00010000:1.0:1789762938.363945:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f7 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.363953:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aac1b1e/0xdf5a7ac59f1775f7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762938.364047:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067535104:12345-192.168.203.37@tcp:4:dd.0 Request processed in 14709us (49498us total) trans 4294968131 rc 0/0 00000100:00100000:1.0:1789762938.364056:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1023 00000100:00100000:1.0:1789762938.364064:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411596313039 time 411596389560 state 0x2 ref 0 00000100:00100000:1.0:1789762938.365389:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762938.365449:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067535360 00000100:00100000:2.0:1789762938.365458:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067535360 00000100:00080000:1.0:1789762938.365499:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762938.365517:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 102 00000100:00100000:2.0:1789762938.365528:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9625880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15693:x1876702067535360:12345-192.168.203.37@tcp:35:dd.0 00000001:00080000:2.0:1789762938.365759:0:6421:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967456 00000100:00100000:2.0:1789762938.365857:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9625880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:15693:x1876702067535360:12345-192.168.203.37@tcp:35:dd.0 Request processed in 330us (468us total) trans 4294967457 rc 0/0 00000100:00100000:2.0:1789762938.365867:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 102 00000020:00080000:2.0:1789762938.365873:0:6421:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1789762938.368038:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074744064:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762938.368090:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762938.368180:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074744064 00000100:00100000:0.0:1789762938.368198:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074744064 00000100:00100000:0.0:1789762938.368328:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 421 00000100:00100000:0.0:1789762938.368360:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9626d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074744064:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789762938.368420:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789762938.368541:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9626d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074744064:12345-0@lo:41:osp-pre-1-0.0 Request processed in 195us (446us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762938.368558:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 421 00000100:00100000:1.0:1789762938.368581:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074744064:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762938.407566:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762938.407678:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067535616 00000100:00100000:0.0:1789762938.407693:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067535616 00000100:00100000:0.0:1789762938.407773:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 353 00000100:00100000:0.0:1789762938.407790:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:15709:x1876702067535616:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:0.0:1789762938.407856:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762938.407896:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca3a6e00/0xdf5a7ac59f1775fe lrc: 2/0,0 mode: --/CR res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cacb expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.408106:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca3a7e00/0xdf5a7ac59f177605 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.408136:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca3a7e00/0xdf5a7ac59f177605 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.408164:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca3a7e00/0xdf5a7ac59f177605 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762938.408187:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x18:0x0]: returning size 104857600 00010000:00010000:0.0:1789762938.408401:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca3a6e00/0xdf5a7ac59f1775fe lrc: 0/0,0 mode: --/CR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cacb expref: 12 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.408428:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca3a7e00/0xdf5a7ac59f177605 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cacb expref: 11 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.408441:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x18:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:0.0:1789762938.408446:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:0.0:1789762938.408451:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ca3a7e00, rc 0) 00000100:00100000:0.0:1789762938.408534:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15709:x1876702067535616:12345-192.168.203.37@tcp:101:dd.0 Request processed in 748us (974us total) trans 4294967458 rc 0/0 00000100:00100000:0.0:1789762938.408545:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 353 00000100:00100000:0.0:1789762938.410025:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762938.410106:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067535744 00000100:00100000:2.0:1789762938.410118:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067535744 00000100:00100000:2.0:1789762938.410181:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 203 00000100:00100000:2.0:1789762938.410194:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067535744:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762938.410212:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762938.410234:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177613 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cad2 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762938.410261:0:9038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 104857600, mtime: 0x6aad9d7a, atime: 0x0, ctime: 0x6aad9d7a, blocks: 0x3200b 00010000:00010000:2.0:1789762938.410281:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=411s v=(0 0) 00010000:00010000:2.0:1789762938.410286:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762938.410297:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177613 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cad2 expref: 6 pid: 9038 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.410316:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177613 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cad2 expref: 6 pid: 9038 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.410330:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8caf000, rc 0) 00000100:00100000:2.0:1789762938.410387:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067535744:12345-192.168.203.37@tcp:101:dd.0 Request processed in 197us (368us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762938.410396:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 203 00000100:00100000:1.0:1789762938.412730:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762938.412773:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067535872 00000100:00100000:3.0:1789762938.412779:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067535872 00000100:00100000:3.0:1789762938.412813:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 366705088325 time 366605107931 state 0x0 ref 1 00000100:00100000:3.0:1789762938.412837:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 366705107931 time 366605107931 state 0x1 ref 1 00000100:00100000:3.0:1789762938.412858:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 366705107931 time 411659936586 state 0x0 ref 1 00000100:00100000:3.0:1789762938.412867:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9625c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067535872:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762938.412886:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17761a lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.412896:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762938.412899:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762938.412903:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17761a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.412911:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17761a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.416381:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17761a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.416391:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17761a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.416398:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17761a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.416403:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762938.416407:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17761a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.416413:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17761a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.416424:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17761a lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.416429:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17761a lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762938.416472:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9625c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067535872:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3606us (3744us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762938.416479:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1024 00000100:00100000:3.0:1789762938.416485:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 366705107931 time 411659936586 state 0x2 ref 0 00000100:00100000:3.0:1789762938.418077:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762938.418130:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067536000 00000100:00100000:1.0:1789762938.418136:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067536000 00000100:00100000:1.0:1789762938.418162:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 366705107931 time 411659936586 state 0x0 ref 1 00000100:00100000:1.0:1789762938.418171:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 2 deadline 411759936586 time 411659936586 state 0x1 ref 1 00000100:00100000:1.0:1789762938.418192:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 411759936586 time 411665271190 state 0x0 ref 1 00000100:00100000:1.0:1789762938.418203:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067536000:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762938.418218:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177621 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.418227:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762938.418230:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762938.418233:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177621 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.418241:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177621 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.420726:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177621 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.420737:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177621 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.420744:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177621 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.420749:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998fe00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762938.420753:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177621 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.420758:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177621 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.420770:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177621 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.420776:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004fda2ab7/0xdf5a7ac59f177621 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762938.420837:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067536000:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2634us (2760us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762938.420845:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1025 00000100:00100000:1.0:1789762938.420851:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 411759936586 time 411665271190 state 0x2 ref 0 00000100:00100000:3.0:1789762938.422318:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762938.422354:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067536128 00000100:00100000:1.0:1789762938.422360:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067536128 00000100:00100000:1.0:1789762938.422382:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 411759936586 time 411665271190 state 0x0 ref 1 00000100:00100000:1.0:1789762938.422391:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 1 deadline 411765271190 time 411665271190 state 0x1 ref 1 00000100:00100000:1.0:1789762938.422411:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411765271190 time 411669490270 state 0x0 ref 1 00000100:00100000:1.0:1789762938.422427:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067536128:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762938.422442:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177628 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.422453:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762938.422459:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762938.422464:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177628 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.422475:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177628 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.424966:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177628 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.424977:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177628 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.424984:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177628 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.424990:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762938.424994:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177628 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.425000:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177628 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.425010:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177628 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.425016:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177628 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762938.425106:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067536128:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2683us (2785us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762938.425115:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1026 00000100:00100000:1.0:1789762938.425124:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411765271190 time 411669490270 state 0x2 ref 0 00000100:00100000:3.0:1789762938.426560:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762938.426596:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067536256 00000100:00100000:1.0:1789762938.426602:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067536256 00000100:00100000:1.0:1789762938.426625:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411765271190 time 411669490270 state 0x0 ref 1 00000100:00100000:1.0:1789762938.426634:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411769490270 time 411669490270 state 0x1 ref 1 00010000:00010000:1.0:1789762938.496087:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762938.522507:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411769490270 time 411769574295 state 0x0 ref 1 00000100:00100000:3.0:1789762938.522533:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067536256:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762938.522568:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17762f lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.522581:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762938.522584:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762938.522588:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17762f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.522599:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17762f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.525389:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17762f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.525402:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17762f lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.525409:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17762f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.525415:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca315400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762938.525419:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17762f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.525430:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17762f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.525444:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17762f lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.525449:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f644340d/0xdf5a7ac59f17762f lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762938.525500:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067536256:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2979us (98941us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762938.525507:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1027 00000100:00100000:1.0:1789762938.525515:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411769490270 time 411769574295 state 0x2 ref 0 00000100:00100000:2.0:1789762938.527420:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762938.527462:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067536384 00000100:00100000:3.0:1789762938.527469:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067536384 00000100:00100000:3.0:1789762938.527504:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411769490270 time 411769574295 state 0x0 ref 1 00000100:00100000:3.0:1789762938.527515:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411869574295 time 411769574295 state 0x1 ref 1 00000100:00100000:2.0:1789762938.622594:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411869574295 time 411869659375 state 0x0 ref 1 00000100:00100000:2.0:1789762938.622622:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067536384:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:2.0:1789762938.622660:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177636 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.622673:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762938.622677:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762938.622682:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177636 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.622694:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177636 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.625936:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177636 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.625948:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177636 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.625954:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177636 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.625960:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762938.625963:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177636 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.625969:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177636 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.625986:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177636 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.625992:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f177636 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762938.626039:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067536384:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3429us (98619us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762938.626046:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1028 00000100:00100000:2.0:1789762938.626055:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411869574295 time 411869659375 state 0x2 ref 0 00000100:00100000:2.0:1789762938.628448:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762938.628507:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067536512 00000100:00100000:1.0:1789762938.628515:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067536512 00000100:00100000:1.0:1789762938.628552:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411869574295 time 411869659375 state 0x0 ref 1 00000100:00100000:1.0:1789762938.628562:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411969659375 time 411869659375 state 0x1 ref 1 00000100:00100000:2.0:1789762938.722697:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411969659375 time 411969762282 state 0x0 ref 1 00000100:00100000:2.0:1789762938.722724:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067536512:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:2.0:1789762938.722762:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17763d lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.722775:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762938.722779:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762938.722784:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17763d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.722794:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17763d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.725595:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17763d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.725607:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17763d lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.725623:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17763d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.725632:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafc00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762938.725638:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17763d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.725648:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17763d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.725668:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17763d lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.725674:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000dac8a32/0xdf5a7ac59f17763d lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762938.725722:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067536512:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3010us (97275us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762938.725729:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1029 00000100:00100000:1.0:1789762938.725736:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411969659375 time 411969762282 state 0x2 ref 0 00000100:00100000:3.0:1789762938.727575:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762938.727604:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067536640 00000100:00100000:1.0:1789762938.727612:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067536640 00000100:00100000:1.0:1789762938.727645:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 411969659375 time 411969762282 state 0x0 ref 1 00000100:00100000:1.0:1789762938.727658:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412069762282 time 411969762282 state 0x1 ref 1 00000100:00100000:1.0:1789762938.822746:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412069762282 time 412069813297 state 0x0 ref 1 00000100:00100000:1.0:1789762938.822770:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067536640:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762938.822803:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177644 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.822832:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762938.822836:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762938.822842:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177644 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.822854:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177644 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.825701:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177644 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.825712:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177644 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.825720:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177644 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.825730:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762938.825733:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177644 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.825740:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177644 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.825752:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177644 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.825757:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177644 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762938.825805:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067536640:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3045us (98231us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762938.825828:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1030 00000100:00100000:1.0:1789762938.825837:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412069762282 time 412069813297 state 0x2 ref 0 00000100:00100000:3.0:1789762938.827664:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762938.827739:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067536768 00000100:00100000:2.0:1789762938.827747:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067536768 00000100:00100000:2.0:1789762938.827781:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412069762282 time 412069813297 state 0x0 ref 1 00000100:00100000:2.0:1789762938.827792:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412169813297 time 412069813297 state 0x1 ref 1 00000100:00100000:3.0:1789762938.922873:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412169813297 time 412169939453 state 0x0 ref 1 00000100:00100000:3.0:1789762938.922898:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067536768:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762938.922951:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f17764b lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.922965:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762938.922969:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762938.922973:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f17764b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.922985:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f17764b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.925650:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f17764b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.925659:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f17764b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.925666:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f17764b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.925673:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e0eb6400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762938.925677:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f17764b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.925684:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f17764b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.925697:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f17764b lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762938.925702:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f17764b lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762938.925749:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067536768:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2862us (98085us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762938.925756:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1031 00000100:00100000:3.0:1789762938.925764:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412169813297 time 412169939453 state 0x2 ref 0 00000100:00100000:0.0:1789762938.927550:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762938.927608:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067536896 00000100:00100000:1.0:1789762938.927618:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067536896 00000100:00100000:1.0:1789762938.927665:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412169813297 time 412169939453 state 0x0 ref 1 00000100:00100000:1.0:1789762938.927676:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412269939453 time 412169939453 state 0x1 ref 1 00000100:00100000:3.0:1789762939.022971:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412269939453 time 412270032803 state 0x0 ref 1 00000100:00100000:3.0:1789762939.023012:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067536896:12345-192.168.203.37@tcp:3:dd.0 00000100:00080000:1.0:1789762939.023037:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762939.023118:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177652 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.023145:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762939.023149:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762939.023155:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177652 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.023169:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177652 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.026178:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177652 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.026187:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177652 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.026194:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177652 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.026200:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e0eb6400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762939.026204:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177652 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.026210:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177652 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.026226:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177652 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.026236:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177652 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762939.026295:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067536896:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3303us (98744us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762939.026304:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1032 00000100:00100000:3.0:1789762939.026314:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412269939453 time 412270032803 state 0x2 ref 0 00000100:00100000:3.0:1789762939.028164:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762939.028235:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067537024 00000100:00100000:1.0:1789762939.028245:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067537024 00000100:00100000:1.0:1789762939.028289:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412269939453 time 412270032803 state 0x0 ref 1 00000100:00100000:1.0:1789762939.028301:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412370032803 time 412270032803 state 0x1 ref 1 00000100:00100000:1.0:1789762939.123035:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412370032803 time 412370098030 state 0x0 ref 1 00000100:00100000:1.0:1789762939.123061:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067537024:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762939.123100:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177659 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.123117:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762939.123121:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762939.123127:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177659 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.123141:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177659 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.125794:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177659 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.125806:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177659 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.125830:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177659 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.125840:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762939.125845:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177659 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.125855:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177659 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.125874:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177659 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.125883:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177659 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762939.125993:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067537024:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2938us (97827us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762939.126005:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1033 00000100:00100000:1.0:1789762939.126015:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412370032803 time 412370098030 state 0x2 ref 0 00000100:00100000:3.0:1789762939.127946:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762939.127981:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067537152 00000100:00100000:1.0:1789762939.127990:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067537152 00000100:00100000:1.0:1789762939.128033:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412370032803 time 412370098030 state 0x0 ref 1 00000100:00100000:1.0:1789762939.128047:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412470098030 time 412370098030 state 0x1 ref 1 00000100:00100000:3.0:1789762939.223123:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412470098030 time 412470189359 state 0x0 ref 1 00000100:00100000:3.0:1789762939.223155:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067537152:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762939.223191:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177660 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.223207:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762939.223212:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762939.223218:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177660 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.223234:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177660 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.226037:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177660 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.226047:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177660 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.226054:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177660 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.226059:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37e0eb6400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762939.226062:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177660 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.226068:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177660 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.226082:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177660 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.226090:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d0cff6b5/0xdf5a7ac59f177660 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762939.226156:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067537152:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3011us (98239us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762939.226164:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1034 00000100:00100000:1.0:1789762939.226171:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412470098030 time 412470189359 state 0x2 ref 0 00000100:00100000:0.0:1789762939.227995:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762939.228037:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067537280 00000100:00100000:3.0:1789762939.228044:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067537280 00000100:00100000:3.0:1789762939.228078:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412470098030 time 412470189359 state 0x0 ref 1 00000100:00100000:3.0:1789762939.228088:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412570189359 time 412470189359 state 0x1 ref 1 00000100:00100000:2.0:1789762939.323203:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412570189359 time 412570263987 state 0x0 ref 1 00000100:00100000:2.0:1789762939.323229:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067537280:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:2.0:1789762939.323266:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f177667 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.323281:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762939.323284:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762939.323289:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f177667 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.323301:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f177667 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.325991:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f177667 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.326002:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f177667 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.326010:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f177667 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.326016:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762939.326020:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f177667 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.326026:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f177667 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.326041:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f177667 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.326046:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ba37f696/0xdf5a7ac59f177667 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762939.326096:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067537280:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2875us (98102us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762939.326103:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1035 00000100:00100000:1.0:1789762939.326110:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412570189359 time 412570263987 state 0x2 ref 0 00000100:00100000:0.0:1789762939.328017:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762939.328053:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067537408 00000100:00100000:1.0:1789762939.328061:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067537408 00000100:00100000:1.0:1789762939.328098:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412570189359 time 412570263987 state 0x0 ref 1 00000100:00100000:1.0:1789762939.328109:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412670263987 time 412570263987 state 0x1 ref 1 00000100:00100000:1.0:1789762939.392077:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eb3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074744192:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762939.392136:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762939.392188:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074744192 00000100:00100000:2.0:1789762939.392196:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074744192 00000100:00100000:2.0:1789762939.392239:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 422 00000100:00100000:2.0:1789762939.392247:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9624700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074744192:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762939.392281:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:2.0:1789762939.392324:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9624700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074744192:12345-0@lo:41:osp-pre-0-1.0 Request processed in 78us (187us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762939.392329:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 422 00000100:00100000:1.0:1789762939.392364:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eb3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074744192:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762939.423324:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412670263987 time 412670390169 state 0x0 ref 1 00000100:00100000:3.0:1789762939.423349:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067537408:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762939.423383:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17766e lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.423397:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762939.423400:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762939.423405:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17766e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.423415:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17766e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.427485:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17766e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.427496:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17766e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.427504:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17766e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.427509:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762939.427512:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17766e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.427519:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17766e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.427532:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17766e lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.427537:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17766e lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762939.427580:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067537408:12345-192.168.203.37@tcp:3:dd.0 Request processed in 4241us (99564us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762939.427586:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1036 00000100:00100000:3.0:1789762939.427594:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412670263987 time 412670390169 state 0x2 ref 0 00000100:00100000:3.0:1789762939.430532:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762939.430581:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067537536 00000100:00100000:1.0:1789762939.430589:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067537536 00000100:00100000:1.0:1789762939.430623:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412670263987 time 412670390169 state 0x0 ref 1 00000100:00100000:1.0:1789762939.430633:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412770390169 time 412670390169 state 0x1 ref 1 00010000:00010000:1.0:1789762939.520121:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762939.523364:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412770390169 time 412770432992 state 0x0 ref 1 00000100:00100000:1.0:1789762939.523389:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067537536:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762939.523426:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177675 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.523440:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762939.523443:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762939.523448:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177675 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.523461:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177675 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.526445:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177675 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.526457:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177675 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.526467:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177675 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.526476:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762939.526481:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177675 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.526493:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177675 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.526513:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177675 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.526519:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177675 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762939.526569:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067537536:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3191us (96036us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762939.526576:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1037 00000100:00100000:1.0:1789762939.526584:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412770390169 time 412770432992 state 0x2 ref 0 00000100:00100000:2.0:1789762939.528482:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762939.528526:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067537664 00000100:00100000:1.0:1789762939.528534:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067537664 00000100:00100000:1.0:1789762939.528568:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412770390169 time 412770432992 state 0x0 ref 1 00000100:00100000:1.0:1789762939.528578:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412870432992 time 412770432992 state 0x1 ref 1 00000100:00100000:3.0:1789762939.623452:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412870432992 time 412870519509 state 0x0 ref 1 00000100:00100000:3.0:1789762939.623478:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067537664:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762939.623514:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17767c lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.623528:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762939.623531:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762939.623536:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17767c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.623547:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17767c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.626616:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17767c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.626627:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17767c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.626634:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17767c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.626639:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762939.626643:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17767c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.626649:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17767c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.626666:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17767c lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.626672:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17767c lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762939.626718:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067537664:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3250us (98236us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762939.626724:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1038 00000100:00100000:3.0:1789762939.626732:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412870432992 time 412870519509 state 0x2 ref 0 00000100:00100000:0.0:1789762939.628637:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762939.628668:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067537792 00000100:00100000:3.0:1789762939.628676:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067537792 00000100:00100000:3.0:1789762939.628712:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412870432992 time 412870519509 state 0x0 ref 1 00000100:00100000:3.0:1789762939.628722:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412970519509 time 412870519509 state 0x1 ref 1 00000100:00100000:3.0:1789762939.723525:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412970519509 time 412970589650 state 0x0 ref 1 00000100:00100000:3.0:1789762939.723555:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067537792:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762939.723592:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177683 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.723606:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762939.723610:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762939.723614:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177683 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.723625:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177683 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.726532:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177683 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.726541:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177683 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.726548:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177683 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.726553:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762939.726557:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177683 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.726563:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177683 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.726577:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177683 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.726583:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177683 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762939.726628:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067537792:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3087us (97992us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762939.726635:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1039 00000100:00100000:3.0:1789762939.726642:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412970519509 time 412970589650 state 0x2 ref 0 00000100:00100000:2.0:1789762939.728673:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762939.728730:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067537920 00000100:00100000:1.0:1789762939.728738:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067537920 00000100:00100000:1.0:1789762939.728774:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 412970519509 time 412970589650 state 0x0 ref 1 00000100:00100000:1.0:1789762939.728784:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413070589650 time 412970589650 state 0x1 ref 1 00000100:00100000:3.0:1789762939.823620:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413070589650 time 413070686294 state 0x0 ref 1 00000100:00100000:3.0:1789762939.823648:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067537920:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762939.823684:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17768a lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.823698:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762939.823702:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762939.823707:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17768a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.823719:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17768a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.826666:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17768a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.826675:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17768a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.826686:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17768a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.826690:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762939.826694:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17768a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.826700:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17768a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.826711:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17768a lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.826716:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17768a lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762939.826763:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067537920:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3127us (98090us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762939.826770:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1040 00000100:00100000:3.0:1789762939.826777:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413070589650 time 413070686294 state 0x2 ref 0 00000100:00100000:0.0:1789762939.828626:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762939.828659:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067538048 00000100:00100000:3.0:1789762939.828667:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067538048 00000100:00100000:3.0:1789762939.828702:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413070589650 time 413070686294 state 0x0 ref 1 00000100:00100000:3.0:1789762939.828716:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413170686294 time 413070686294 state 0x1 ref 1 00000001:00080000:2.0:1789762939.910889:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967458 is committed 00000020:00080000:2.0:1789762939.910910:0:7221:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 13 batchid updated 00000020:00080000:3.0:1789762939.910932:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 13 00000004:00080000:3.0:1789762939.910988:0:7233:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37c96abd80 with version 29 00000020:00080000:3.0:1789762939.911002:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 13 cancel update log [0x200000401:0x2:0x0].13: rc = 0 00000004:00080000:0.0:1789762939.911017:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 29 rpc_version 29 00000020:00100000:0.0:1789762939.911049:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762939.911057:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37ee83d180 x1876702074744448/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762939.911077:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ee83d180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074744448:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762939.911140:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762939.911174:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074744448 00000100:00100000:2.0:1789762939.911181:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074744448 00000100:00100000:0.0:1789762939.911190:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d176f300 going to sleep for 11 seconds 00000100:00100000:2.0:1789762939.911223:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 423 00000100:00100000:2.0:1789762939.911232:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:7224:x1876702074744448:12345-0@lo:1000:osp_up0-1.0 00000020:00080000:3.0:1789762939.911574:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 13 cancel update log [0x240000400:0x2:0x0].13: rc = 0 00000020:00080000:3.0:1789762939.911585:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 13 00000100:00100000:2.0:1789762939.911653:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074744448:12345-0@lo:1000:osp_up0-1.0 Request processed in 421us (513us total) trans 4294967446 rc 0/0 00000100:00100000:2.0:1789762939.911661:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 423 00000100:00100000:0.0:1789762939.911695:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ee83d180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074744448:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1789762939.923703:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413170686294 time 413170773844 state 0x0 ref 1 00000100:00100000:1.0:1789762939.923718:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067538048:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762939.923745:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177691 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.923758:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762939.923762:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762939.923767:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177691 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.923778:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177691 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.926502:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177691 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.926514:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177691 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.926521:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177691 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.926527:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762939.926531:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177691 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.926537:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177691 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.926551:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177691 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.926556:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f177691 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762939.926603:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067538048:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2888us (97978us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762939.926610:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1041 00000100:00100000:1.0:1789762939.926618:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413170686294 time 413170773844 state 0x2 ref 0 00000100:00100000:2.0:1789762939.928712:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762939.928745:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067538176 00000100:00100000:3.0:1789762939.928752:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067538176 00000100:00100000:3.0:1789762939.928786:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413170686294 time 413170773844 state 0x0 ref 1 00000100:00100000:3.0:1789762939.928796:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413270773844 time 413170773844 state 0x1 ref 1 00000100:00100000:2.0:1789762940.023843:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413270773844 time 413270908087 state 0x0 ref 1 00000100:00100000:2.0:1789762940.023868:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067538176:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:2.0:1789762940.023904:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000061c40564/0xdf5a7ac59f177698 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762940.023934:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762940.023938:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762940.023943:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c40564/0xdf5a7ac59f177698 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762940.023960:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c40564/0xdf5a7ac59f177698 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.027102:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000061c40564/0xdf5a7ac59f177698 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.027113:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c40564/0xdf5a7ac59f177698 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.027120:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c40564/0xdf5a7ac59f177698 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.027126:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c98b4000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762940.027130:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c40564/0xdf5a7ac59f177698 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.027136:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c40564/0xdf5a7ac59f177698 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.027151:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c40564/0xdf5a7ac59f177698 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.027157:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c40564/0xdf5a7ac59f177698 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762940.027214:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067538176:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3356us (98502us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762940.027222:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1042 00000100:00100000:1.0:1789762940.027233:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413270773844 time 413270908087 state 0x2 ref 0 00000100:00100000:0.0:1789762940.029129:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762940.029165:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067538304 00000100:00100000:3.0:1789762940.029173:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067538304 00000100:00100000:3.0:1789762940.029209:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413270773844 time 413270908087 state 0x0 ref 1 00000100:00100000:3.0:1789762940.029219:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413370908087 time 413270908087 state 0x1 ref 1 00000100:00100000:1.0:1789762940.123941:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413370908087 time 413370990415 state 0x0 ref 1 00000100:00100000:1.0:1789762940.123963:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067538304:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762940.123999:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17769f lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.124014:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762940.124018:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762940.124022:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17769f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.124034:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17769f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.126959:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17769f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.126970:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17769f lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.126981:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17769f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.126987:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762940.126991:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17769f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.126997:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17769f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.127012:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17769f lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.127017:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17769f lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762940.127068:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067538304:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3112us (97939us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762940.127077:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1043 00000100:00100000:1.0:1789762940.127084:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413370908087 time 413370990415 state 0x2 ref 0 00000100:00100000:0.0:1789762940.129145:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762940.129185:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067538432 00000100:00100000:1.0:1789762940.129196:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067538432 00000100:00100000:1.0:1789762940.129233:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413370908087 time 413370990415 state 0x0 ref 1 00000100:00100000:1.0:1789762940.129246:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413470990415 time 413370990415 state 0x1 ref 1 00000100:00100000:1.0:1789762940.223995:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413470990415 time 413471058502 state 0x0 ref 1 00000100:00100000:1.0:1789762940.224021:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067538432:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762940.224059:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776a6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.224073:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762940.224076:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762940.224081:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776a6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.224093:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776a6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.226975:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776a6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.226987:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776a6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.226994:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776a6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.227003:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762940.227007:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776a6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.227013:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776a6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.227027:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776a6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.227032:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776a6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762940.227082:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067538432:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3073us (97937us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762940.227089:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1044 00000100:00100000:1.0:1789762940.227097:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413470990415 time 413471058502 state 0x2 ref 0 00000100:00100000:2.0:1789762940.228969:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762940.229002:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067538560 00000100:00100000:1.0:1789762940.229011:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067538560 00000100:00100000:1.0:1789762940.229046:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413470990415 time 413471058502 state 0x0 ref 1 00000100:00100000:1.0:1789762940.229057:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413571058502 time 413471058502 state 0x1 ref 1 00000100:00100000:1.0:1789762940.324058:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413571058502 time 413571124690 state 0x0 ref 1 00000100:00100000:1.0:1789762940.324090:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067538560:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762940.324127:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776ad lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.324140:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762940.324144:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762940.324148:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776ad lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.324159:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776ad lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.326789:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776ad lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.326819:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776ad lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.326827:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776ad lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.326832:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762940.326835:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776ad lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.326841:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776ad lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.326855:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776ad lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.326860:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f1776ad lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762940.326914:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067538560:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2839us (97945us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762940.326958:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1045 00000100:00100000:1.0:1789762940.326966:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413571058502 time 413571124690 state 0x2 ref 0 00000100:00100000:2.0:1789762940.328971:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762940.329003:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067538688 00000100:00100000:1.0:1789762940.329012:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067538688 00000100:00100000:1.0:1789762940.329045:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413571058502 time 413571124690 state 0x0 ref 1 00000100:00100000:1.0:1789762940.329055:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413671124690 time 413571124690 state 0x1 ref 1 00000100:00100000:0.0:1789762940.359718:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762940.359744:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762940.359766:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762940.359768:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067538816 00000100:00100000:1.0:1789762940.359775:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067538816 00000100:00100000:0.0:1789762940.359787:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762940.359818:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067538944 00000100:00100000:3.0:1789762940.359824:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067538944 00000100:00100000:0.0:1789762940.359828:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762940.359840:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 387 00000100:00100000:2.0:1789762940.359848:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067539072 00000100:00100000:1.0:1789762940.359855:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15c4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067538816:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762940.359855:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067539072 00000100:00100000:0.0:1789762940.359856:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067539200 00000100:00100000:0.0:1789762940.359863:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067539200 00000100:00100000:3.0:1789762940.359877:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067539328 00000100:00100000:3.0:1789762940.359882:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067539328 00000100:00100000:2.0:1789762940.359906:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 628 00000100:00100000:0.0:1789762940.359915:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 629 00000100:00100000:1.0:1789762940.359915:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15c4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067538816:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 64us (198us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762940.359941:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067538944:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762940.359955:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 630 00000100:00100000:1.0:1789762940.359957:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 387 00000100:00100000:0.0:1789762940.359962:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067539072:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762940.359972:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067539200:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762940.359981:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067538944:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 67us (234us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762940.359987:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 628 00000100:00100000:0.0:1789762940.360003:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067539072:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 47us (235us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762940.360008:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 631 00000100:00100000:3.0:1789762940.360016:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067539200:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 48us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762940.360024:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067539328:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762940.360025:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 629 00000100:00100000:3.0:1789762940.360040:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 630 00000100:00100000:2.0:1789762940.360058:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067539328:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 41us (232us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762940.360064:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 631 00000100:00080000:3.0:1789762940.360070:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762940.360075:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762940.424139:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413671124690 time 413671205570 state 0x0 ref 1 00000100:00100000:3.0:1789762940.424166:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067538688:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762940.424204:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776b4 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.424218:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762940.424221:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762940.424226:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776b4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.424237:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776b4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.426758:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776b4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.426767:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776b4 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.426773:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776b4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.426780:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762940.426784:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776b4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.426790:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776b4 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.426814:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776b4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.426819:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776b4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762940.426872:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067538688:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2718us (97901us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762940.426880:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1046 00000100:00100000:3.0:1789762940.426887:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413671124690 time 413671205570 state 0x2 ref 0 00000100:00100000:1.0:1789762940.429088:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762940.429122:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067539456 00000100:00100000:3.0:1789762940.429130:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067539456 00000100:00100000:3.0:1789762940.429166:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413671124690 time 413671205570 state 0x0 ref 1 00000100:00100000:3.0:1789762940.429177:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413771205570 time 413671205570 state 0x1 ref 1 00000100:00100000:3.0:1789762940.524205:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413771205570 time 413771271784 state 0x0 ref 1 00000100:00100000:3.0:1789762940.524235:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9627800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067539456:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762940.524274:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776bb lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.524288:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762940.524291:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762940.524296:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776bb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.524307:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776bb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.527578:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776bb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.527590:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776bb lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.527600:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776bb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.527609:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762940.527614:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776bb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.527623:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776bb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.527638:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776bb lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.527643:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776bb lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762940.527691:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9627800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067539456:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3470us (98604us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762940.527698:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1047 00000100:00100000:3.0:1789762940.527706:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413771205570 time 413771271784 state 0x2 ref 0 00000100:00100000:2.0:1789762940.529791:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762940.529842:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067539584 00000100:00100000:1.0:1789762940.529850:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067539584 00000100:00100000:1.0:1789762940.529888:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413771205570 time 413771271784 state 0x0 ref 1 00000100:00100000:1.0:1789762940.529898:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413871271784 time 413771271784 state 0x1 ref 1 00010000:00010000:2.0:1789762940.544058:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762940.624333:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413871271784 time 413871393067 state 0x0 ref 1 00000100:00100000:3.0:1789762940.624366:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067539584:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762940.624416:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776c2 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.624437:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762940.624441:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762940.624448:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776c2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.624465:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776c2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.628038:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776c2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.628052:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776c2 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.628060:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776c2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.628065:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762940.628069:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776c2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.628077:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776c2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.628098:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776c2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.628106:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776c2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762940.628167:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067539584:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3812us (98373us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762940.628177:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1048 00000100:00100000:3.0:1789762940.628189:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413871271784 time 413871393067 state 0x2 ref 0 00000100:00100000:2.0:1789762940.630467:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762940.630500:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067539712 00000100:00100000:3.0:1789762940.630508:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067539712 00000100:00100000:3.0:1789762940.630549:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413871271784 time 413871393067 state 0x0 ref 1 00000100:00100000:3.0:1789762940.630560:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413971393067 time 413871393067 state 0x1 ref 1 00000100:00100000:1.0:1789762940.724420:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413971393067 time 413971484910 state 0x0 ref 1 00000100:00100000:1.0:1789762940.724449:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067539712:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762940.724485:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776c9 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.724499:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762940.724502:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762940.724506:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776c9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.724518:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776c9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.727644:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776c9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.727654:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776c9 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.727665:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776c9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.727670:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762940.727673:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776c9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.727679:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776c9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.727691:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776c9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.727696:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776c9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762940.727743:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067539712:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3308us (97278us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762940.727750:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1049 00000100:00100000:1.0:1789762940.727757:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413971393067 time 413971484910 state 0x2 ref 0 00000100:00100000:1.0:1789762940.729829:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762940.729872:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067539840 00000100:00100000:3.0:1789762940.729881:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067539840 00000100:00100000:3.0:1789762940.729944:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 413971393067 time 413971484910 state 0x0 ref 1 00000100:00100000:3.0:1789762940.729955:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414071484910 time 413971484910 state 0x1 ref 1 00000100:00100000:3.0:1789762940.824493:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414071484910 time 414071559302 state 0x0 ref 1 00000100:00100000:3.0:1789762940.824519:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9625f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067539840:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762940.824555:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776d0 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.824569:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762940.824572:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762940.824576:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776d0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.824588:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776d0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.827741:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776d0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.827752:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776d0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.827759:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776d0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.827764:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762940.827767:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776d0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.827773:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776d0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.827786:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776d0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.827791:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776d0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762940.827858:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9625f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067539840:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3350us (98029us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762940.827866:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1050 00000100:00100000:3.0:1789762940.827873:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414071484910 time 414071559302 state 0x2 ref 0 00000100:00100000:1.0:1789762940.829903:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762940.829967:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067539968 00000100:00100000:3.0:1789762940.829975:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067539968 00000100:00100000:3.0:1789762940.830010:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414071484910 time 414071559302 state 0x0 ref 1 00000100:00100000:3.0:1789762940.830021:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414171559302 time 414071559302 state 0x1 ref 1 00000100:00100000:1.0:1789762940.924579:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414171559302 time 414171645345 state 0x0 ref 1 00000100:00100000:1.0:1789762940.924610:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9626300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067539968:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762940.924648:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776d7 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.924662:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762940.924665:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762940.924670:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776d7 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.924682:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776d7 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.927594:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776d7 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.927605:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776d7 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.927611:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776d7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.927616:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762940.927620:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776d7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.927626:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776d7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.927639:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776d7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.927648:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f1776d7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762940.927697:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9626300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067539968:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3099us (97794us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762940.927704:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1051 00000100:00100000:3.0:1789762940.927711:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414171559302 time 414171645345 state 0x2 ref 0 00000100:00100000:0.0:1789762940.929579:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762940.929623:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067540096 00000100:00100000:1.0:1789762940.929631:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067540096 00000100:00100000:1.0:1789762940.929663:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414171559302 time 414171645345 state 0x0 ref 1 00000100:00100000:1.0:1789762940.929673:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414271645345 time 414171645345 state 0x1 ref 1 00000100:00100000:1.0:1789762941.024678:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414271645345 time 414271738914 state 0x0 ref 1 00000100:00100000:1.0:1789762941.024731:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067540096:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762941.024773:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776de lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789762941.024786:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762941.024789:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.024794:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.024814:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776de lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.024831:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776de lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.028661:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776de lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.028680:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776de lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.028691:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776de lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.028718:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762941.028729:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776de lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.028749:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776de lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.028793:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776de lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.028837:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776de lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762941.028962:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067540096:12345-192.168.203.37@tcp:3:dd.0 Request processed in 4223us (99341us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762941.028975:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1052 00000100:00100000:1.0:1789762941.028987:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414271645345 time 414271738914 state 0x2 ref 0 00000100:00100000:2.0:1789762941.031384:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762941.031426:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067540224 00000100:00100000:3.0:1789762941.031435:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067540224 00000100:00100000:3.0:1789762941.031474:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414271645345 time 414271738914 state 0x0 ref 1 00000100:00100000:3.0:1789762941.031485:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414371738914 time 414271738914 state 0x1 ref 1 00000100:00100000:1.0:1789762941.124775:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414371738914 time 414371837676 state 0x0 ref 1 00000100:00100000:1.0:1789762941.124811:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067540224:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762941.124879:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776e5 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.124908:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.124917:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.124928:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776e5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.124991:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776e5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.128289:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776e5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.128300:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776e5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.128311:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776e5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.128317:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762941.128320:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776e5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.128326:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776e5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.128338:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776e5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.128343:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776e5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762941.128391:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067540224:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3598us (97008us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762941.128398:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1053 00000100:00100000:1.0:1789762941.128406:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414371738914 time 414371837676 state 0x2 ref 0 00000100:00100000:0.0:1789762941.131045:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762941.131129:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067540352 00000100:00100000:3.0:1789762941.131143:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067540352 00000100:00100000:3.0:1789762941.131201:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414371738914 time 414371837676 state 0x0 ref 1 00000100:00100000:3.0:1789762941.131221:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414471837676 time 414371837676 state 0x1 ref 1 00000100:00100000:3.0:1789762941.224847:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414471837676 time 414471911966 state 0x0 ref 1 00000100:00100000:3.0:1789762941.224874:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067540352:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762941.224911:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776ec lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.224943:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762941.224947:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762941.224951:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776ec lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.224962:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776ec lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.227665:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776ec lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.227676:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776ec lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.227683:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776ec lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.227689:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762941.227693:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776ec lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.227700:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776ec lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.227730:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776ec lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.227739:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1776ec lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762941.227808:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067540352:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2935us (96757us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762941.227819:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1054 00000100:00100000:3.0:1789762941.227830:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414471837676 time 414471911966 state 0x2 ref 0 00000100:00100000:2.0:1789762941.229897:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762941.229955:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067540480 00000100:00100000:1.0:1789762941.229962:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067540480 00000100:00100000:1.0:1789762941.229997:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414471837676 time 414471911966 state 0x0 ref 1 00000100:00100000:1.0:1789762941.230011:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414571911966 time 414471911966 state 0x1 ref 1 00000100:00100000:1.0:1789762941.324911:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414571911966 time 414571977114 state 0x0 ref 1 00000100:00100000:1.0:1789762941.324957:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067540480:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762941.325005:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776f3 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.325019:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.325023:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.325027:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776f3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.325039:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776f3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.327826:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776f3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.327834:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776f3 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.327840:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776f3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.327859:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762941.327862:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776f3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.327868:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776f3 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.327880:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776f3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.327885:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776f3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762941.327961:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067540480:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3004us (98056us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762941.327970:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1055 00000100:00100000:1.0:1789762941.327978:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414571911966 time 414571977114 state 0x2 ref 0 00000100:00100000:2.0:1789762941.329961:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762941.329992:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067540608 00000100:00100000:1.0:1789762941.329999:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067540608 00000100:00100000:1.0:1789762941.330030:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414571911966 time 414571977114 state 0x0 ref 1 00000100:00100000:1.0:1789762941.330041:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414671977114 time 414571977114 state 0x1 ref 1 00000100:00100000:1.0:1789762941.424981:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414671977114 time 414672045311 state 0x0 ref 1 00000100:00100000:1.0:1789762941.425010:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067540608:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762941.425046:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776fa lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.425060:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.425064:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.425068:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776fa lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.425079:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776fa lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.428192:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776fa lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.428205:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776fa lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.428216:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776fa lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.428225:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762941.428231:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776fa lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.428248:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776fa lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.428266:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776fa lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.428273:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f1776fa lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762941.428323:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067540608:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3326us (98362us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762941.428330:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1056 00000100:00100000:1.0:1789762941.428338:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414671977114 time 414672045311 state 0x2 ref 0 00000100:00100000:0.0:1789762941.430525:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762941.430586:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067540736 00000100:00100000:3.0:1789762941.430594:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067540736 00000100:00100000:3.0:1789762941.430632:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414671977114 time 414672045311 state 0x0 ref 1 00000100:00100000:3.0:1789762941.430654:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414772045311 time 414672045311 state 0x1 ref 1 00000100:00100000:1.0:1789762941.440026:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074744576:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762941.440065:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eb0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074744832:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762941.440070:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074744704:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762941.440085:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762941.440117:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762941.440145:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074744576 00000100:00100000:2.0:1789762941.440150:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074744576 00000100:00100000:3.0:1789762941.440162:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d921d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074745344:0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.440163:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d921d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074745088:0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.440175:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074744832 00000100:00100000:0.0:1789762941.440180:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074744832 00000100:00100000:2.0:1789762941.440195:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 258 00000100:00100000:3.0:1789762941.440200:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762941.440208:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9624000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074744576:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762941.440215:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762941.440224:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d921f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074745856:0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.440241:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 259 00002000:00000024:2.0:1789762941.440241:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3754950656 avail 3752853504 00000100:00100000:1.0:1789762941.440245:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d921ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074745600:0@lo:400:kworker.0 00000100:00100000:3.0:1789762941.440254:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762941.440260:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074744832:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762941.440279:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3754950656 avail 3752853504 00000100:00100000:3.0:1789762941.440285:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9e77800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074746368:0@lo:400:kworker.0 00000100:00100000:2.0:1789762941.440291:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9624000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074744576:12345-0@lo:13:osp-pre-0-1.0 Request processed in 85us (209us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762941.440299:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762941.440304:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 258 00000100:00100000:1.0:1789762941.440332:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d921e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074746112:0@lo:400:kworker.0 00000100:00100000:2.0:1789762941.440334:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074745344 00000100:00100000:0.0:1789762941.440340:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074744832:12345-0@lo:13:osp-pre-0-0.0 Request processed in 86us (221us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762941.440341:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074745344 00000100:00100000:3.0:1789762941.440345:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762941.440353:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 259 00000100:00100000:1.0:1789762941.440366:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762941.440379:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 424 00000100:00100000:0.0:1789762941.440389:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074745088 00000100:00100000:2.0:1789762941.440390:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3427:x1876702074745344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.440393:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074745088 00000100:00100000:3.0:1789762941.440398:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eb0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074744832:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762941.440422:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1789762941.440424:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074744576:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762941.440430:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074745856 00000100:00100000:3.0:1789762941.440434:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d921c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074745728:0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.440435:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074745856 00000100:00100000:2.0:1789762941.440441:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3427:x1876702074745344:12345-0@lo:400:kworker.0 Request processed in 51us (240us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762941.440447:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 424 00000100:00100000:1.0:1789762941.440455:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:0.0:1789762941.440461:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074746112 00000100:00100000:0.0:1789762941.440465:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074746112 00000100:00100000:2.0:1789762941.440467:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074745600 00000100:00100000:1.0:1789762941.440470:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d921ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074744960:0@lo:400:kworker.0 00000100:00100000:3.0:1789762941.440472:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762941.440475:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074745600 00000100:00100000:0.0:1789762941.440488:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074745728 00000100:00100000:0.0:1789762941.440493:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074745728 00000100:00100000:3.0:1789762941.440496:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d921e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074746240:0@lo:400:kworker.0 00000100:00100000:2.0:1789762941.440520:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:0.0:1789762941.440525:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 632 00000100:00100000:1.0:1789762941.440527:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762941.440536:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9627480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074745600:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.440539:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9627b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074745088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762941.440545:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762941.440565:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d921dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074745472:0@lo:400:kworker.0 00000100:00100000:3.0:1789762941.440595:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d921d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074745344:0@lo:400:kworker.0 00000100:00100000:2.0:1789762941.440600:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9627480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074745600:12345-0@lo:400:kworker.0 Request processed in 69us (305us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762941.440608:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9627b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3425:x1876702074745088:12345-0@lo:400:kworker.0 Request processed in 72us (394us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762941.440613:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00100000:1.0:1789762941.440619:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762941.440622:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074746368 00000100:00100000:0.0:1789762941.440626:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 632 00000100:00100000:3.0:1789762941.440628:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074746368 00000100:00100000:0.0:1789762941.440642:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074745472 00000100:00100000:1.0:1789762941.440647:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d921df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074745984:0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.440649:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074745472 00000100:00100000:2.0:1789762941.440651:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762941.440672:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074746240 00000100:00100000:3.0:1789762941.440678:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074746240 00000100:00100000:0.0:1789762941.440679:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 633 00000100:00100000:2.0:1789762941.440683:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eb3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074745216:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762941.440690:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762941.440694:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074745856:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.440715:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9e75f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074746496:0@lo:400:kworker.0 00000100:00100000:2.0:1789762941.440733:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762941.440734:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 259 00000100:00100000:3.0:1789762941.440748:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074746368:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.440759:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3427:x1876702074745856:12345-0@lo:400:kworker.0 Request processed in 71us (505us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762941.440766:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 633 00000100:00100000:1.0:1789762941.440782:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762941.440783:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074745984 00000100:00100000:2.0:1789762941.440787:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 260 00000100:00100000:0.0:1789762941.440788:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074745984 00000100:00100000:3.0:1789762941.440807:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074746368:12345-0@lo:400:kworker.0 Request processed in 49us (452us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762941.440822:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 259 00000100:00100000:2.0:1789762941.440823:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074746240:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.440837:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d921d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3425:1876702074745088:0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.440838:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 634 00000100:00100000:3.0:1789762941.440845:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074746496 00000100:00100000:0.0:1789762941.440858:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9625180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074746112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762941.440859:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074746496 00000100:00100000:1.0:1789762941.440865:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d921ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074745600:0@lo:400:kworker.0 00000100:00100000:2.0:1789762941.440887:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074746240:12345-0@lo:400:kworker.0 Request processed in 90us (350us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762941.440893:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 261 00000100:00100000:2.0:1789762941.440900:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 260 00000100:00100000:3.0:1789762941.440905:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074746496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.440911:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d921e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074746112:0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.440912:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9625180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074746112:12345-0@lo:400:kworker.0 Request processed in 60us (545us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762941.440955:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 634 00000100:00100000:2.0:1789762941.440956:0:8055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074744704 00000100:00100000:2.0:1789762941.440961:0:8055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074744704 00000100:00100000:1.0:1789762941.440969:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074745216 00000100:00100000:1.0:1789762941.440974:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074745216 00000100:00100000:0.0:1789762941.440981:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 635 00000100:00100000:3.0:1789762941.440983:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074746496:12345-0@lo:400:kworker.0 Request processed in 79us (200us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762941.440991:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 261 00000100:00100000:0.0:1789762941.440995:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074745728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762941.441004:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 260 00000100:00100000:1.0:1789762941.441014:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 261 00000100:00100000:2.0:1789762941.441021:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074744704:12345-0@lo:13:osp-pre-1-0.0 00000100:00080000:3.0:1789762941.441023:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762941.441028:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074745216:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762941.441032:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074745728:12345-0@lo:400:kworker.0 Request processed in 42us (563us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762941.441032:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789762941.441037:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762941.441041:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 635 00000100:00080000:3.0:1789762941.441041:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00000024:2.0:1789762941.441053:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00002000:00000024:1.0:1789762941.441054:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762941.441062:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 636 00000100:00100000:0.0:1789762941.441075:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 637 00000100:00100000:3.0:1789762941.441076:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9626680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074745472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.441089:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9624a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074745984:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.441111:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074745216:12345-0@lo:13:osp-pre-1-1.0 Request processed in 86us (389us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762941.441121:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074744704:12345-0@lo:13:osp-pre-1-0.0 Request processed in 106us (473us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762941.441123:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 261 00000100:00100000:3.0:1789762941.441136:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9626680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074745472:12345-0@lo:400:kworker.0 Request processed in 64us (523us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762941.441137:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 260 00000100:00100000:0.0:1789762941.441139:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9624a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3425:x1876702074745984:12345-0@lo:400:kworker.0 Request processed in 52us (450us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762941.441152:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 636 00000100:00100000:0.0:1789762941.441158:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 637 00000100:00100000:1.0:1789762941.441162:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074744960 00000100:00100000:1.0:1789762941.441169:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074744960 00000100:00100000:2.0:1789762941.441192:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074744704:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762941.441207:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eb3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074745216:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762941.441222:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 388 00000100:00100000:1.0:1789762941.441234:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15c4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074744960:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.441240:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d921c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074745728:0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.441262:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d921e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074746240:0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.441279:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d921f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3427:1876702074745856:0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.441286:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15c4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074744960:12345-0@lo:400:kworker.0 Request processed in 54us (761us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762941.441292:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 388 00000100:00100000:0.0:1789762941.441296:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9e77800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074746368:0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.441330:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d921ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074744960:0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.441341:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d921dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074745472:0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.441347:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d921df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3425:1876702074745984:0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.441352:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9e75f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074746496:0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.525051:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414772045311 time 414772117469 state 0x0 ref 1 00000100:00100000:1.0:1789762941.525077:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067540736:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762941.525115:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177701 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.525129:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.525132:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.525138:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177701 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.525149:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177701 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.527846:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177701 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.527857:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177701 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.527864:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177701 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.527871:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762941.527875:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177701 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.527881:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177701 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.527894:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177701 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.527900:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177701 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762941.527976:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067540736:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2908us (97451us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762941.527983:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1057 00000100:00100000:1.0:1789762941.527991:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414772045311 time 414772117469 state 0x2 ref 0 00000100:00100000:0.0:1789762941.529867:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762941.529907:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067540864 00000100:00100000:1.0:1789762941.529930:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067540864 00000100:00100000:1.0:1789762941.529966:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414772045311 time 414772117469 state 0x0 ref 1 00000100:00100000:1.0:1789762941.529977:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414872117469 time 414772117469 state 0x1 ref 1 00010000:00010000:0.0:1789762941.568104:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762941.625144:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414872117469 time 414872210321 state 0x0 ref 1 00000100:00100000:3.0:1789762941.625172:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e77800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067540864:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762941.625210:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177708 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.625223:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762941.625227:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762941.625231:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177708 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.625251:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177708 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.627957:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177708 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.627966:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177708 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.627977:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177708 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.627982:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762941.627986:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177708 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.627992:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177708 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.628007:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177708 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.628012:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177708 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762941.628059:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e77800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067540864:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2900us (98194us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762941.628066:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1058 00000100:00100000:3.0:1789762941.628073:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414872117469 time 414872210321 state 0x2 ref 0 00000100:00100000:2.0:1789762941.629835:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762941.629891:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067540992 00000100:00100000:1.0:1789762941.629900:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067540992 00000100:00100000:1.0:1789762941.629964:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414872117469 time 414872210321 state 0x0 ref 1 00000100:00100000:1.0:1789762941.629975:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414972210321 time 414872210321 state 0x1 ref 1 00000100:00100000:1.0:1789762941.725203:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414972210321 time 414972268856 state 0x0 ref 1 00000100:00100000:1.0:1789762941.725228:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067540992:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762941.725265:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17770f lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.725278:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.725282:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.725286:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17770f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.725305:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17770f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.728206:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17770f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.728222:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17770f lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.728230:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17770f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.728236:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762941.728250:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17770f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.728258:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17770f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.728272:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17770f lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.728277:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17770f lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762941.728336:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067540992:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3112us (98498us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762941.728347:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1059 00000100:00100000:1.0:1789762941.728357:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414972210321 time 414972268856 state 0x2 ref 0 00000100:00100000:2.0:1789762941.730131:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762941.730169:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067541120 00000100:00100000:1.0:1789762941.730178:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067541120 00000100:00100000:1.0:1789762941.730211:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 414972210321 time 414972268856 state 0x0 ref 1 00000100:00100000:1.0:1789762941.730221:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415072268856 time 414972268856 state 0x1 ref 1 00000100:00100000:3.0:1789762941.825313:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415072268856 time 415072379597 state 0x0 ref 1 00000100:00100000:3.0:1789762941.825342:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067541120:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762941.825387:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177716 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.825401:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762941.825404:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762941.825408:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177716 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.825421:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177716 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.829008:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177716 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.829019:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177716 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.829026:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177716 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.829031:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762941.829034:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177716 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.829041:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177716 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.829059:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177716 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.829065:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177716 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762941.829112:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067541120:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3784us (98982us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762941.829119:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1060 00000100:00100000:3.0:1789762941.829127:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415072268856 time 415072379597 state 0x2 ref 0 00000100:00100000:2.0:1789762941.831041:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762941.831074:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067541248 00000100:00100000:3.0:1789762941.831081:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067541248 00000100:00100000:3.0:1789762941.831116:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415072268856 time 415072379597 state 0x0 ref 1 00000100:00100000:3.0:1789762941.831126:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415172379597 time 415072379597 state 0x1 ref 1 00000100:00100000:3.0:1789762941.925379:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415172379597 time 415172444856 state 0x0 ref 1 00000100:00100000:3.0:1789762941.925406:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067541248:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762941.925442:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17771d lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.925456:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762941.925460:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762941.925465:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17771d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.925476:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17771d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.928668:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17771d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.928677:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17771d lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.928683:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17771d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.928688:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762941.928691:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17771d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.928698:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17771d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.928711:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17771d lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.928717:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17771d lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762941.928765:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067541248:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3372us (97725us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762941.928773:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1061 00000100:00100000:3.0:1789762941.928781:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415172379597 time 415172444856 state 0x2 ref 0 00000100:00100000:2.0:1789762941.930975:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762941.931025:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067541376 00000100:00100000:1.0:1789762941.931032:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067541376 00000100:00100000:1.0:1789762941.931079:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415172379597 time 415172444856 state 0x0 ref 1 00000100:00100000:1.0:1789762941.931090:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415272444856 time 415172444856 state 0x1 ref 1 00000100:00100000:3.0:1789762942.025498:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415272444856 time 415272562944 state 0x0 ref 1 00000100:00100000:3.0:1789762942.025524:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067541376:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762942.025560:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177724 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.025574:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762942.025578:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762942.025582:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177724 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.025594:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177724 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.028403:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177724 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.028411:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177724 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.028422:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177724 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.028428:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762942.028432:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177724 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.028438:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177724 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.028453:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177724 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.028459:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177724 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762942.028507:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067541376:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2994us (97533us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762942.028512:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1062 00000100:00100000:3.0:1789762942.028519:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415272444856 time 415272562944 state 0x2 ref 0 00000100:00100000:0.0:1789762942.030594:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762942.030627:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067541504 00000100:00100000:3.0:1789762942.030634:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067541504 00000100:00100000:3.0:1789762942.030668:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415272444856 time 415272562944 state 0x0 ref 1 00000100:00100000:3.0:1789762942.030682:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415372562944 time 415272562944 state 0x1 ref 1 00000100:00100000:1.0:1789762942.125569:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415372562944 time 415372635457 state 0x0 ref 1 00000100:00100000:1.0:1789762942.125608:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e76300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067541504:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762942.125645:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17772b lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.125659:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762942.125663:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762942.125668:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17772b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.125679:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17772b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.128672:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17772b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.128682:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17772b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.128689:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17772b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.128695:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762942.128698:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17772b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.128704:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17772b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.128717:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17772b lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.128722:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f17772b lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762942.128771:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e76300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067541504:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3179us (98178us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762942.128778:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1063 00000100:00100000:1.0:1789762942.128786:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415372562944 time 415372635457 state 0x2 ref 0 00000100:00100000:0.0:1789762942.130885:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762942.130950:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067541632 00000100:00100000:3.0:1789762942.130958:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067541632 00000100:00100000:3.0:1789762942.131002:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415372562944 time 415372635457 state 0x0 ref 1 00000100:00100000:3.0:1789762942.131018:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415472635457 time 415372635457 state 0x1 ref 1 00000100:00100000:3.0:1789762942.225638:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415472635457 time 415472700351 state 0x0 ref 1 00000100:00100000:3.0:1789762942.225670:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e76680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067541632:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762942.225712:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177732 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.225728:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762942.225732:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762942.225739:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177732 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.225754:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177732 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.229160:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177732 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.229177:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177732 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.229186:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177732 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.229196:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762942.229201:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177732 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.229211:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177732 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.229230:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177732 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.229240:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177732 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762942.229297:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e76680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067541632:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3640us (98411us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762942.229307:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1064 00000100:00100000:3.0:1789762942.229318:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415472635457 time 415472700351 state 0x2 ref 0 00000100:00100000:2.0:1789762942.231338:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762942.231380:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067541760 00000100:00100000:3.0:1789762942.231389:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067541760 00000100:00100000:3.0:1789762942.231426:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415472635457 time 415472700351 state 0x0 ref 1 00000100:00100000:3.0:1789762942.231440:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415572700351 time 415472700351 state 0x1 ref 1 00000100:00100000:1.0:1789762942.325733:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415572700351 time 415572799434 state 0x0 ref 1 00000100:00100000:1.0:1789762942.325759:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067541760:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762942.325818:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177739 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.325837:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762942.325841:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762942.325846:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177739 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.325857:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177739 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.328735:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177739 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.328746:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177739 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.328752:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177739 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.328758:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762942.328761:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177739 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.328767:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177739 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.328779:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177739 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.328784:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cc28616/0xdf5a7ac59f177739 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762942.328853:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067541760:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3103us (97515us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762942.328860:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1065 00000100:00100000:3.0:1789762942.328868:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415572700351 time 415572799434 state 0x2 ref 0 00000100:00100000:2.0:1789762942.330804:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762942.330860:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067541888 00000100:00100000:1.0:1789762942.330870:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067541888 00000100:00100000:1.0:1789762942.330907:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415572700351 time 415572799434 state 0x0 ref 1 00000100:00100000:1.0:1789762942.330936:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415672799434 time 415572799434 state 0x1 ref 1 00000100:00100000:1.0:1789762942.425799:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415672799434 time 415672854009 state 0x0 ref 1 00000100:00100000:1.0:1789762942.425829:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067541888:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762942.425869:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177740 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.425883:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762942.425886:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762942.425890:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177740 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.425906:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177740 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.428785:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177740 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.428808:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177740 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.428817:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177740 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.428824:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762942.428828:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177740 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.428837:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177740 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.428855:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177740 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.428865:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177740 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762942.428957:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067541888:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3140us (98166us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762942.428968:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1066 00000100:00100000:1.0:1789762942.428979:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415672799434 time 415672854009 state 0x2 ref 0 00000100:00100000:0.0:1789762942.430876:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762942.430910:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067542016 00000100:00100000:3.0:1789762942.430939:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067542016 00000100:00100000:3.0:1789762942.430979:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415672799434 time 415672854009 state 0x0 ref 1 00000100:00100000:3.0:1789762942.430991:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415772854009 time 415672854009 state 0x1 ref 1 00000100:00100000:1.0:1789762942.525886:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415772854009 time 415772944649 state 0x0 ref 1 00000100:00100000:1.0:1789762942.525915:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e74000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067542016:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762942.525976:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177747 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.525990:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762942.525994:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762942.525999:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177747 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.526009:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177747 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.528962:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177747 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.528974:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177747 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.528986:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177747 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.528992:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762942.528997:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177747 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.529007:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177747 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.529025:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177747 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.529034:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177747 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762942.529088:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e74000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067542016:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3185us (98212us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762942.529095:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1067 00000100:00100000:1.0:1789762942.529103:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415772854009 time 415772944649 state 0x2 ref 0 00000100:00100000:2.0:1789762942.531628:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762942.531673:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067542144 00000100:00100000:3.0:1789762942.531682:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067542144 00000100:00100000:3.0:1789762942.531724:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415772854009 time 415772944649 state 0x0 ref 1 00000100:00100000:3.0:1789762942.531739:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415872944649 time 415772944649 state 0x1 ref 1 00010000:00010000:1.0:1789762942.592082:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762942.625959:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415872944649 time 415873021671 state 0x0 ref 1 00000100:00100000:3.0:1789762942.625989:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067542144:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762942.626027:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17774e lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.626041:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762942.626046:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762942.626050:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17774e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.626061:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17774e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.628734:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17774e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.628744:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17774e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.628750:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17774e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.628756:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762942.628760:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17774e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.628766:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17774e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.628781:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17774e lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.628802:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17774e lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762942.628853:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067542144:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2878us (97225us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762942.628860:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1068 00000100:00100000:3.0:1789762942.628868:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415872944649 time 415873021671 state 0x2 ref 0 00000100:00100000:2.0:1789762942.630874:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762942.630938:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067542272 00000100:00100000:1.0:1789762942.630946:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067542272 00000100:00100000:1.0:1789762942.630980:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415872944649 time 415873021671 state 0x0 ref 1 00000100:00100000:1.0:1789762942.630996:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415973021671 time 415873021671 state 0x1 ref 1 00000100:00100000:1.0:1789762942.726018:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415973021671 time 415973082026 state 0x0 ref 1 00000100:00100000:1.0:1789762942.726047:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067542272:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762942.726087:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177755 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.726102:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762942.726105:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762942.726109:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177755 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.726121:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177755 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.728703:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177755 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.728713:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177755 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.728719:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177755 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.728733:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762942.728738:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177755 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.728747:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177755 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.728765:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177755 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.728774:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177755 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762942.728835:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067542272:12345-192.168.203.37@tcp:3:dd.0 Request processed in 2802us (97962us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762942.728844:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1069 00000100:00100000:1.0:1789762942.728852:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415973021671 time 415973082026 state 0x2 ref 0 00000100:00100000:2.0:1789762942.730930:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762942.730977:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067542400 00000100:00100000:1.0:1789762942.730985:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067542400 00000100:00100000:1.0:1789762942.731020:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 415973021671 time 415973082026 state 0x0 ref 1 00000100:00100000:1.0:1789762942.731030:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416073082026 time 415973082026 state 0x1 ref 1 00000100:00100000:1.0:1789762942.826080:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416073082026 time 416073146311 state 0x0 ref 1 00000100:00100000:1.0:1789762942.826114:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cdd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067542400:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762942.826158:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17775c lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.826173:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762942.826176:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762942.826182:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17775c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.826194:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17775c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.829144:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17775c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.829159:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17775c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.829168:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17775c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.829175:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762942.829179:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17775c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.829189:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17775c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.829209:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17775c lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.829218:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17775c lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762942.829277:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cdd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067542400:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3178us (98361us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762942.829284:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1070 00000100:00100000:1.0:1789762942.829295:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416073082026 time 416073146311 state 0x2 ref 0 00000100:00100000:2.0:1789762942.831181:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762942.831222:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067542528 00000100:00100000:3.0:1789762942.831231:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067542528 00000100:00100000:3.0:1789762942.831271:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416073082026 time 416073146311 state 0x0 ref 1 00000100:00100000:3.0:1789762942.831281:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416173146311 time 416073146311 state 0x1 ref 1 00000100:00100000:3.0:1789762942.926138:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416173146311 time 416173204229 state 0x0 ref 1 00000100:00100000:3.0:1789762942.926167:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067542528:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762942.926206:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177763 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.926220:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762942.926224:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762942.926229:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177763 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.926239:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177763 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.929021:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177763 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.929037:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177763 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.929048:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177763 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.929057:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762942.929062:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177763 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.929073:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177763 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.929092:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177763 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.929101:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177763 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762942.929160:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067542528:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3005us (97976us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762942.929172:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1071 00000100:00100000:3.0:1789762942.929184:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416173146311 time 416173204229 state 0x2 ref 0 00000100:00100000:0.0:1789762942.931015:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762942.931060:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067542656 00000100:00100000:1.0:1789762942.931069:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067542656 00000100:00100000:1.0:1789762942.931106:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416173146311 time 416173204229 state 0x0 ref 1 00000100:00100000:1.0:1789762942.931116:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416273204229 time 416173204229 state 0x1 ref 1 00000100:00100000:3.0:1789762943.026247:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416273204229 time 416273311434 state 0x0 ref 1 00000100:00100000:3.0:1789762943.026292:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e77b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067542656:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762943.026328:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17776a lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789762943.026340:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762943.026343:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762943.026348:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762943.026353:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17776a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.026365:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17776a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.029267:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17776a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.029283:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17776a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.029293:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17776a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.029303:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762943.029310:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17776a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.029321:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17776a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.029341:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17776a lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.029351:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17776a lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762943.029413:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e77b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067542656:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3143us (98396us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762943.029424:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1072 00000100:00100000:3.0:1789762943.029435:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416273204229 time 416273311434 state 0x2 ref 0 00000100:00100000:0.0:1789762943.031362:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762943.031408:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067542784 00000100:00100000:1.0:1789762943.031416:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067542784 00000100:00100000:1.0:1789762943.031456:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416273204229 time 416273311434 state 0x0 ref 1 00000100:00100000:1.0:1789762943.031467:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416373311434 time 416273311434 state 0x1 ref 1 00000100:00100000:1.0:1789762943.126303:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416373311434 time 416373368354 state 0x0 ref 1 00000100:00100000:1.0:1789762943.126339:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e77480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067542784:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762943.126375:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177771 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.126388:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762943.126392:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762943.126397:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177771 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.126408:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177771 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.129627:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177771 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.129643:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177771 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.129653:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177771 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.129663:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762943.129668:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177771 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.129677:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177771 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.129692:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177771 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.129698:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177771 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762943.129748:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e77480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067542784:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3423us (98387us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762943.129755:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1073 00000100:00100000:1.0:1789762943.129763:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416373311434 time 416373368354 state 0x2 ref 0 00000100:00100000:2.0:1789762943.131734:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762943.131794:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067542912 00000100:00100000:3.0:1789762943.131804:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067542912 00000100:00100000:3.0:1789762943.131853:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416373311434 time 416373368354 state 0x0 ref 1 00000100:00100000:3.0:1789762943.131868:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416473368354 time 416373368354 state 0x1 ref 1 00000100:00100000:3.0:1789762943.226378:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416473368354 time 416473440491 state 0x0 ref 1 00000100:00100000:3.0:1789762943.226407:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067542912:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762943.226444:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177778 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.226462:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762943.226466:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762943.226473:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177778 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.226489:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177778 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.229516:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177778 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.229532:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177778 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.229543:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177778 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.229552:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762943.229557:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177778 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.229566:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177778 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.229599:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177778 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.229607:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177778 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762943.229691:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067542912:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3295us (97955us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762943.229700:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1074 00000100:00100000:3.0:1789762943.229711:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416473368354 time 416473440491 state 0x2 ref 0 00000100:00100000:0.0:1789762943.231743:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762943.231776:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067543040 00000100:00100000:3.0:1789762943.231795:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067543040 00000100:00100000:3.0:1789762943.231842:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416473368354 time 416473440491 state 0x0 ref 1 00000100:00100000:3.0:1789762943.231852:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416573440491 time 416473440491 state 0x1 ref 1 00000100:00100000:3.0:1789762943.326473:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416573440491 time 416573525391 state 0x0 ref 1 00000100:00100000:3.0:1789762943.326513:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e75180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067543040:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762943.326565:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17777f lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.326585:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762943.326591:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762943.326597:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17777f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.326618:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17777f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.329951:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17777f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.329965:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17777f lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.329974:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17777f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.329981:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762943.329985:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17777f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.329993:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17777f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.330012:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17777f lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.330018:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17777f lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762943.330086:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e75180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067543040:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3590us (98342us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762943.330097:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1075 00000100:00100000:3.0:1789762943.330107:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416573440491 time 416573525391 state 0x2 ref 0 00000100:00100000:0.0:1789762943.332181:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762943.332256:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067543168 00000100:00100000:1.0:1789762943.332269:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067543168 00000100:00100000:1.0:1789762943.332326:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416573440491 time 416573525391 state 0x0 ref 1 00000100:00100000:1.0:1789762943.332340:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416673525391 time 416573525391 state 0x1 ref 1 00000100:00100000:1.0:1789762943.426554:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416673525391 time 416673613234 state 0x0 ref 1 00000100:00100000:1.0:1789762943.426588:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e74a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067543168:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762943.426640:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177786 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.426658:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762943.426663:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762943.426669:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177786 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.426683:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177786 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.429933:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177786 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.429946:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177786 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.429953:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177786 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.429959:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762943.429963:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177786 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.429978:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177786 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.429996:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177786 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.430002:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f177786 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762943.430073:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e74a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067543168:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3499us (97892us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762943.430084:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1076 00000100:00100000:1.0:1789762943.430093:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416673525391 time 416673613234 state 0x2 ref 0 00000100:00100000:2.0:1789762943.432036:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762943.432079:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067543296 00000100:00100000:3.0:1789762943.432091:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067543296 00000100:00100000:3.0:1789762943.432142:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416673525391 time 416673613234 state 0x0 ref 1 00000100:00100000:3.0:1789762943.432154:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416773613234 time 416673613234 state 0x1 ref 1 00000100:00100000:1.0:1789762943.488104:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789762943.488139:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074746624:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762943.488229:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762943.488319:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074746624 00000100:00100000:1.0:1789762943.488328:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074746624 00000100:00100000:1.0:1789762943.488387:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 426 00000100:00100000:1.0:1789762943.488397:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074746624:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762943.488679:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762943.488736:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074746624:12345-0@lo:41:osp-pre-1-0.0 Request processed in 341us (509us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762943.488744:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 426 00000100:00100000:3.0:1789762943.488861:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074746624:0@lo:41:osp-pre-1-0.0 00000001:00080000:3.0:1789762943.496135:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967446 is committed 00000100:00100000:1.0:1789762943.526669:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416773613234 time 416773727581 state 0x0 ref 1 00000100:00100000:1.0:1789762943.526700:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067543296:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762943.526753:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17778d lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.526770:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762943.526773:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762943.526795:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17778d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.526816:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17778d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.530070:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17778d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.530087:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17778d lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.530097:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17778d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.530103:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762943.530106:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17778d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.530114:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17778d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.530132:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17778d lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.530137:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae1ecf1c/0xdf5a7ac59f17778d lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762943.530195:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067543296:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3507us (98159us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762943.530204:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1077 00000100:00100000:2.0:1789762943.530212:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416773613234 time 416773727581 state 0x2 ref 0 00000100:00100000:2.0:1789762943.532114:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762943.532158:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067543424 00000100:00100000:3.0:1789762943.532168:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067543424 00000100:00100000:3.0:1789762943.532216:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416773613234 time 416773727581 state 0x0 ref 1 00000100:00100000:3.0:1789762943.532229:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416873727581 time 416773727581 state 0x1 ref 1 00010000:00010000:1.0:1789762943.616203:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762943.626716:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416873727581 time 416873779554 state 0x0 ref 1 00000100:00100000:3.0:1789762943.626742:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067543424:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762943.626805:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177794 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.626829:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762943.626836:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762943.626842:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177794 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.626858:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177794 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.629830:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177794 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.629845:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177794 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.629856:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177794 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.629865:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762943.629872:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177794 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.629883:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177794 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.629904:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177794 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.629943:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177794 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762943.630008:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067543424:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3274us (97892us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762943.630017:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1078 00000100:00100000:3.0:1789762943.630026:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416873727581 time 416873779554 state 0x2 ref 0 00000100:00100000:0.0:1789762943.631979:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762943.632015:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067543552 00000100:00100000:1.0:1789762943.632026:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067543552 00000100:00100000:1.0:1789762943.632072:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416873727581 time 416873779554 state 0x0 ref 1 00000100:00100000:1.0:1789762943.632085:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416973779554 time 416873779554 state 0x1 ref 1 00000100:00100000:2.0:1789762943.726795:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416973779554 time 416973840629 state 0x0 ref 1 00000100:00100000:2.0:1789762943.726825:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e77100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067543552:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:2.0:1789762943.726880:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f17779b lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.726902:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762943.726908:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762943.726933:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f17779b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.726952:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f17779b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.730210:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f17779b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.730228:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f17779b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.730240:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f17779b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.730247:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca314000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762943.730251:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f17779b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.730263:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f17779b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.730281:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f17779b lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.730286:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000184e8f0f/0xdf5a7ac59f17779b lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762943.730342:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e77100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067543552:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3526us (98363us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762943.730350:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1079 00000100:00100000:2.0:1789762943.730360:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416973779554 time 416973840629 state 0x2 ref 0 00000100:00100000:0.0:1789762943.732316:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762943.732347:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067543680 00000100:00100000:2.0:1789762943.732356:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067543680 00000100:00100000:2.0:1789762943.732399:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 416973779554 time 416973840629 state 0x0 ref 1 00000100:00100000:2.0:1789762943.732411:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417073840629 time 416973840629 state 0x1 ref 1 00000100:00100000:1.0:1789762943.826884:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417073840629 time 417073943911 state 0x0 ref 1 00000100:00100000:1.0:1789762943.826947:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e74380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067543680:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762943.826998:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777a2 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.827017:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762943.827021:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762943.827026:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777a2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.827042:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777a2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.830121:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777a2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.830134:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777a2 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.830142:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777a2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.830149:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762943.830153:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777a2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.830161:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777a2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.830179:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777a2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.830185:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777a2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762943.830247:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e74380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067543680:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3342us (97933us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762943.830255:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1080 00000100:00100000:1.0:1789762943.830265:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417073840629 time 417073943911 state 0x2 ref 0 00000100:00100000:0.0:1789762943.832334:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762943.832378:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067543808 00000100:00100000:3.0:1789762943.832388:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067543808 00000100:00100000:3.0:1789762943.832434:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417073840629 time 417073943911 state 0x0 ref 1 00000100:00100000:3.0:1789762943.832447:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417173943911 time 417073943911 state 0x1 ref 1 00000100:00100000:3.0:1789762943.926969:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417173943911 time 417174027181 state 0x0 ref 1 00000100:00100000:3.0:1789762943.927003:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e76a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067543808:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762943.927063:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777a9 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.927082:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762943.927087:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762943.927095:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777a9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.927119:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777a9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.929988:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777a9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.929999:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777a9 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.930005:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777a9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.930010:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762943.930014:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777a9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.930021:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777a9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.930038:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777a9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.930043:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777a9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762943.930112:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e76a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067543808:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3122us (97777us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762943.930124:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1081 00000100:00100000:3.0:1789762943.930136:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417173943911 time 417174027181 state 0x2 ref 0 00000100:00100000:0.0:1789762943.932138:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762943.932194:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067543936 00000100:00100000:2.0:1789762943.932206:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067543936 00000100:00100000:2.0:1789762943.932252:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417173943911 time 417174027181 state 0x0 ref 1 00000100:00100000:2.0:1789762943.932265:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417274027181 time 417174027181 state 0x1 ref 1 00000100:00100000:3.0:1789762944.027066:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417274027181 time 417274125346 state 0x0 ref 1 00000100:00100000:3.0:1789762944.027101:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e74e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067543936:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762944.027156:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777b0 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.027174:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762944.027179:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762944.027185:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777b0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.027199:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777b0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.030133:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777b0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.030144:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777b0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.030156:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777b0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.030163:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762944.030167:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777b0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.030174:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777b0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.030191:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777b0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.030196:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777b0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762944.030256:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e74e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067543936:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3171us (98120us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762944.030264:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1082 00000100:00100000:3.0:1789762944.030274:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417274027181 time 417274125346 state 0x2 ref 0 00000100:00100000:2.0:1789762944.032220:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762944.032269:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067544064 00000100:00100000:1.0:1789762944.032279:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067544064 00000100:00100000:1.0:1789762944.032323:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417274027181 time 417274125346 state 0x0 ref 1 00000100:00100000:1.0:1789762944.032336:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417374125346 time 417274125346 state 0x1 ref 1 00000100:00100000:1.0:1789762944.127124:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417374125346 time 417374185038 state 0x0 ref 1 00000100:00100000:1.0:1789762944.127149:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067544064:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762944.127196:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777b7 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.127214:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762944.127218:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762944.127223:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777b7 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.127236:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777b7 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.130443:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777b7 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.130454:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777b7 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.130462:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777b7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.130467:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762944.130471:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777b7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.130478:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777b7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.130494:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777b7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.130499:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777b7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762944.130555:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067544064:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3414us (98336us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762944.130564:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1083 00000100:00100000:1.0:1789762944.130572:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417374125346 time 417374185038 state 0x2 ref 0 00000100:00100000:2.0:1789762944.132375:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762944.132423:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067544192 00000100:00100000:3.0:1789762944.132434:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067544192 00000100:00100000:3.0:1789762944.132479:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417374125346 time 417374185038 state 0x0 ref 1 00000100:00100000:3.0:1789762944.132496:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417474185038 time 417374185038 state 0x1 ref 1 00000100:00100000:1.0:1789762944.227247:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417474185038 time 417474301002 state 0x0 ref 1 00000100:00100000:1.0:1789762944.227284:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067544192:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762944.227358:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777be lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.227377:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762944.227382:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762944.227387:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777be lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.227401:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777be lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.230130:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777be lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.230145:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777be lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.230155:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777be lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.230164:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762944.230170:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777be lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.230180:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777be lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.230206:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777be lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.230215:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777be lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762944.230277:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067544192:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3010us (97904us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762944.230285:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1084 00000100:00100000:1.0:1789762944.230294:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417474185038 time 417474301002 state 0x2 ref 0 00000100:00100000:0.0:1789762944.232468:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762944.232514:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067544320 00000100:00100000:3.0:1789762944.232523:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067544320 00000100:00100000:3.0:1789762944.232576:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417474185038 time 417474301002 state 0x0 ref 1 00000100:00100000:3.0:1789762944.232615:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417574301002 time 417474301002 state 0x1 ref 1 00000100:00100000:3.0:1789762944.327334:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417574301002 time 417574386770 state 0x0 ref 1 00000100:00100000:3.0:1789762944.327369:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e75500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067544320:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762944.327428:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777c5 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.327452:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762944.327458:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762944.327466:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777c5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.327485:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777c5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.331985:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777c5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.332010:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777c5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.332020:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777c5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.332028:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762944.332033:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777c5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.332044:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777c5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.332073:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777c5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.332088:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777c5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762944.332182:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e75500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067544320:12345-192.168.203.37@tcp:3:dd.0 Request processed in 4822us (99710us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762944.332198:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1085 00000100:00100000:3.0:1789762944.332216:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417574301002 time 417574386770 state 0x2 ref 0 00000100:00100000:0.0:1789762944.334880:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762944.334991:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067544448 00000100:00100000:1.0:1789762944.335008:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067544448 00000100:00100000:1.0:1789762944.335089:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417574301002 time 417574386770 state 0x0 ref 1 00000100:00100000:1.0:1789762944.335113:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417674386770 time 417574386770 state 0x1 ref 1 00000100:00100000:2.0:1789762944.427428:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417674386770 time 417674486379 state 0x0 ref 1 00000100:00100000:2.0:1789762944.427459:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e75880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067544448:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:2.0:1789762944.427516:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006ccf1f65/0xdf5a7ac59f1777cc lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.427535:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762944.427539:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762944.427544:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ccf1f65/0xdf5a7ac59f1777cc lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.427558:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ccf1f65/0xdf5a7ac59f1777cc lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.430729:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006ccf1f65/0xdf5a7ac59f1777cc lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.430745:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ccf1f65/0xdf5a7ac59f1777cc lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.430757:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ccf1f65/0xdf5a7ac59f1777cc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.430792:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a6200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762944.430798:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ccf1f65/0xdf5a7ac59f1777cc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.430809:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ccf1f65/0xdf5a7ac59f1777cc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.430831:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ccf1f65/0xdf5a7ac59f1777cc lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.430839:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ccf1f65/0xdf5a7ac59f1777cc lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762944.430907:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e75880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067544448:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3459us (96030us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762944.430959:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1086 00000100:00100000:2.0:1789762944.430973:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417674386770 time 417674486379 state 0x2 ref 0 00000100:00100000:2.0:1789762944.432788:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762944.432834:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067544576 00000100:00100000:1.0:1789762944.432847:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067544576 00000100:00100000:1.0:1789762944.432903:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417674386770 time 417674486379 state 0x0 ref 1 00000100:00100000:1.0:1789762944.432940:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417774486379 time 417674486379 state 0x1 ref 1 00000100:00100000:0.0:1789762944.512179:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074746752:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789762944.512274:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762944.512326:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074746752 00000100:00100000:3.0:1789762944.512340:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074746752 00000100:00100000:3.0:1789762944.512398:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 427 00000100:00100000:3.0:1789762944.512409:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e75c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074746752:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789762944.512457:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:3.0:1789762944.512510:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e75c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074746752:12345-0@lo:41:osp-pre-0-1.0 Request processed in 103us (236us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762944.512518:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 427 00000100:00100000:0.0:1789762944.512539:0:3426:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967446 gen 2 00000004:00080000:0.0:1789762944.512546:0:3426:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967446 committed transno 4294967446 00000100:00100000:0.0:1789762944.512575:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074746752:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762944.527478:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417774486379 time 417774539678 state 0x0 ref 1 00000100:00100000:1.0:1789762944.527510:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067544576:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762944.527551:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777d3 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.527569:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762944.527574:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762944.527583:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777d3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.527602:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777d3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.530699:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777d3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.530711:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777d3 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.530717:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777d3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.530722:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762944.530726:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777d3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.530733:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777d3 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.530751:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777d3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.530758:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777d3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762944.530837:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067544576:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3339us (98049us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762944.530845:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1087 00000100:00100000:1.0:1789762944.530853:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417774486379 time 417774539678 state 0x2 ref 0 00000100:00100000:0.0:1789762944.532790:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762944.532841:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067544704 00000100:00100000:3.0:1789762944.532851:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067544704 00000100:00100000:3.0:1789762944.532904:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417774486379 time 417774539678 state 0x0 ref 1 00000100:00100000:3.0:1789762944.532937:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417874539678 time 417774539678 state 0x1 ref 1 00000100:00100000:3.0:1789762944.627552:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417874539678 time 417874612971 state 0x0 ref 1 00000100:00100000:3.0:1789762944.627583:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e76d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067544704:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762944.627630:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777da lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.627666:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762944.627671:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762944.627678:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777da lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.627693:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777da lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.631347:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777da lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.631357:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777da lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.631364:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777da lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.631370:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762944.631374:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777da lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.631381:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777da lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.631396:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777da lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.631402:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777da lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762944.631458:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e76d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067544704:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3888us (98670us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762944.631466:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1088 00000100:00100000:3.0:1789762944.631475:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417874539678 time 417874612971 state 0x2 ref 0 00000100:00100000:0.0:1789762944.633963:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762944.634017:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067544832 00000100:00100000:2.0:1789762944.634027:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067544832 00000100:00100000:2.0:1789762944.634072:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417874539678 time 417874612971 state 0x0 ref 1 00000100:00100000:2.0:1789762944.634084:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417974612971 time 417874612971 state 0x1 ref 1 00010000:00010000:2.0:1789762944.640067:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762944.727720:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417974612971 time 417974776556 state 0x0 ref 1 00000100:00100000:3.0:1789762944.727759:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e74700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067544832:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762944.727853:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777e1 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.727873:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762944.727877:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762944.727883:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777e1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.727917:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777e1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.731018:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777e1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.731033:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777e1 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.731044:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777e1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.731054:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762944.731067:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777e1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.731078:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777e1 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.731101:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777e1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.731110:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1777e1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762944.731180:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e74700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067544832:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3438us (97216us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762944.731188:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1089 00000100:00100000:3.0:1789762944.731196:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417974612971 time 417974776556 state 0x2 ref 0 00000100:00100000:2.0:1789762944.733057:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762944.733109:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067544960 00000100:00100000:1.0:1789762944.733121:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067544960 00000100:00100000:1.0:1789762944.733182:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 417974612971 time 417974776556 state 0x0 ref 1 00000100:00100000:1.0:1789762944.733198:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418074776556 time 417974776556 state 0x1 ref 1 00000100:00100000:1.0:1789762944.827809:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418074776556 time 418074860939 state 0x0 ref 1 00000100:00100000:1.0:1789762944.827850:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067544960:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762944.827910:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777e8 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.827956:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762944.827962:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762944.827971:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777e8 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.827992:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777e8 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.831568:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777e8 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.831589:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777e8 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.831601:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777e8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.831609:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762944.831616:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777e8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.831627:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777e8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.831649:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777e8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.831658:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002c93332/0xdf5a7ac59f1777e8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762944.831723:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067544960:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3889us (98665us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762944.831732:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1090 00000100:00100000:3.0:1789762944.831743:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418074776556 time 418074860939 state 0x2 ref 0 00000100:00100000:2.0:1789762944.833939:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762944.833963:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067545088 00000100:00100000:3.0:1789762944.833974:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067545088 00000100:00100000:3.0:1789762944.834022:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418074776556 time 418074860939 state 0x0 ref 1 00000100:00100000:3.0:1789762944.834035:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418174860939 time 418074860939 state 0x1 ref 1 00000100:00100000:1.0:1789762944.927986:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418174860939 time 418175039996 state 0x0 ref 1 00000100:00100000:1.0:1789762944.928023:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067545088:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762944.928088:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777ef lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.928112:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762944.928116:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762944.928123:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777ef lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.928139:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777ef lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.931294:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777ef lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.931306:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777ef lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.931319:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777ef lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.931325:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5861a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762944.931329:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777ef lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.931336:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777ef lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.931352:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777ef lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762944.931357:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777ef lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762944.931417:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067545088:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3410us (97479us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762944.931427:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1091 00000100:00100000:1.0:1789762944.931436:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418174860939 time 418175039996 state 0x2 ref 0 00000100:00100000:2.0:1789762944.933206:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762944.933266:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067545216 00000100:00100000:3.0:1789762944.933278:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067545216 00000100:00100000:3.0:1789762944.933338:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418174860939 time 418175039996 state 0x0 ref 1 00000100:00100000:3.0:1789762944.933361:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418275039996 time 418175039996 state 0x1 ref 1 00000100:00100000:1.0:1789762945.028021:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418275039996 time 418275084705 state 0x0 ref 1 00000100:00100000:1.0:1789762945.028060:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067545216:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762945.028121:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777f6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.028142:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00000100:00080000:0.0:1789762945.028144:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762945.028146:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762945.028155:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777f6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.028170:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777f6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.031380:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777f6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.031395:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777f6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.031407:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777f6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.031417:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5861a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762945.031423:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777f6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.031434:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777f6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.031457:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777f6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.031464:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cdcbe65a/0xdf5a7ac59f1777f6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762945.031528:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067545216:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3486us (98321us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762945.031541:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1092 00000100:00100000:0.0:1789762945.031553:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418275039996 time 418275084705 state 0x2 ref 0 00000100:00100000:1.0:1789762945.033465:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762945.033508:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067545344 00000100:00100000:3.0:1789762945.033521:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067545344 00000100:00100000:3.0:1789762945.033575:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418275039996 time 418275084705 state 0x0 ref 1 00000100:00100000:3.0:1789762945.033593:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418375084705 time 418275084705 state 0x1 ref 1 00000100:00100000:2.0:1789762945.128114:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418375084705 time 418375169491 state 0x0 ref 1 00000100:00100000:2.0:1789762945.128144:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067545344:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:2.0:1789762945.128191:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007d04630e/0xdf5a7ac59f1777fd lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.128208:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762945.128212:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762945.128217:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d04630e/0xdf5a7ac59f1777fd lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.128231:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d04630e/0xdf5a7ac59f1777fd lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.131410:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007d04630e/0xdf5a7ac59f1777fd lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.131422:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d04630e/0xdf5a7ac59f1777fd lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.131430:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d04630e/0xdf5a7ac59f1777fd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.131442:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d920c800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762945.131447:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d04630e/0xdf5a7ac59f1777fd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.131454:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d04630e/0xdf5a7ac59f1777fd lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.131471:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d04630e/0xdf5a7ac59f1777fd lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.131477:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d04630e/0xdf5a7ac59f1777fd lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762945.131531:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067545344:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3396us (98066us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762945.131538:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1093 00000100:00100000:2.0:1789762945.131547:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418375084705 time 418375169491 state 0x2 ref 0 00000100:00100000:2.0:1789762945.133403:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762945.133447:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067545472 00000100:00100000:1.0:1789762945.133457:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067545472 00000100:00100000:1.0:1789762945.133503:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418375084705 time 418375169491 state 0x0 ref 1 00000100:00100000:1.0:1789762945.133521:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418475169491 time 418375169491 state 0x1 ref 1 00000100:00100000:1.0:1789762945.228233:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418475169491 time 418475292590 state 0x0 ref 1 00000100:00100000:1.0:1789762945.228266:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067545472:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762945.228316:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177804 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.228334:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762945.228338:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762945.228345:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177804 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.228358:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177804 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.231517:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177804 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.231542:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177804 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.231553:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177804 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.231562:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762945.231567:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177804 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.231578:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177804 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.231605:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177804 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.231615:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005a140c8b/0xdf5a7ac59f177804 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762945.231709:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067545472:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3454us (98304us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762945.231724:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1094 00000100:00100000:1.0:1789762945.231738:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418475169491 time 418475292590 state 0x2 ref 0 00000100:00100000:2.0:1789762945.234870:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762945.234957:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067545600 00000100:00100000:0.0:1789762945.234968:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067545600 00000100:00100000:0.0:1789762945.235040:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418475169491 time 418475292590 state 0x0 ref 1 00000100:00100000:0.0:1789762945.235060:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418575292590 time 418475292590 state 0x1 ref 1 00000100:00100000:3.0:1789762945.328377:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418575292590 time 418575418976 state 0x0 ref 1 00000100:00100000:3.0:1789762945.328412:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067545600:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762945.328462:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17780b lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.328481:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762945.328485:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762945.328492:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17780b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.328511:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17780b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.331689:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17780b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.331714:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17780b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.331728:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17780b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.331735:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762945.331740:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17780b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.331749:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17780b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.331782:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17780b lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.331792:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17780b lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762945.331862:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067545600:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3460us (97000us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762945.331874:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1095 00000100:00100000:3.0:1789762945.331887:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418575292590 time 418575418976 state 0x2 ref 0 00000100:00100000:2.0:1789762945.333790:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762945.333855:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067545728 00000100:00100000:1.0:1789762945.333867:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067545728 00000100:00100000:1.0:1789762945.333947:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418575292590 time 418575418976 state 0x0 ref 1 00000100:00100000:1.0:1789762945.333966:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418675418976 time 418575418976 state 0x1 ref 1 00000100:00100000:3.0:1789762945.428802:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418675418976 time 418675834101 state 0x0 ref 1 00000100:00100000:3.0:1789762945.428854:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067545728:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762945.428982:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177812 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.429010:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762945.429016:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762945.429026:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177812 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.429048:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177812 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.432886:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177812 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.432910:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177812 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.432943:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177812 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.432954:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762945.432960:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177812 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.432974:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177812 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.432999:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177812 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.433010:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177812 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762945.433095:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067545728:12345-192.168.203.37@tcp:3:dd.0 Request processed in 4254us (99301us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762945.433109:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1096 00000100:00100000:3.0:1789762945.433124:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418675418976 time 418675834101 state 0x2 ref 0 00000100:00100000:0.0:1789762945.435775:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762945.435834:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067545856 00000100:00100000:1.0:1789762945.435846:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067545856 00000100:00100000:1.0:1789762945.435908:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418675418976 time 418675834101 state 0x0 ref 1 00000100:00100000:1.0:1789762945.435948:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418775834101 time 418675834101 state 0x1 ref 1 00000100:00100000:2.0:1789762945.479843:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762945.479894:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762945.479912:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067546112 00000100:00100000:3.0:1789762945.479948:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067546112 00000100:00100000:0.0:1789762945.479954:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067545984 00000100:00100000:0.0:1789762945.479961:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067545984 00000100:00100000:2.0:1789762945.479981:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00080000:1.0:1789762945.480001:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762945.480017:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762945.480024:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 389 00000100:00100000:3.0:1789762945.480030:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067546240 00000100:00100000:3.0:1789762945.480036:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067546240 00000100:00100000:1.0:1789762945.480038:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067546368 00000100:00100000:0.0:1789762945.480040:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15c5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067545984:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762945.480054:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067546368 00000100:00100000:2.0:1789762945.480058:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762945.480077:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067546496 00000100:00100000:3.0:1789762945.480081:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067546496 00000100:00100000:1.0:1789762945.480100:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 638 00000100:00100000:1.0:1789762945.480112:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067546112:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762945.480118:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15c5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067545984:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 83us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762945.480123:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 639 00000100:00100000:0.0:1789762945.480137:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 389 00000100:00100000:3.0:1789762945.480140:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 640 00000100:00100000:2.0:1789762945.480143:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067546240:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762945.480156:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067546368:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762945.480157:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067546112:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 45us (320us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762945.480175:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 638 00000100:00100000:0.0:1789762945.480187:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 641 00000100:00100000:2.0:1789762945.480190:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067546240:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 50us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762945.480199:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067546368:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 48us (182us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762945.480203:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067546496:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762945.480204:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 639 00000100:00080000:1.0:1789762945.480205:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762945.480212:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762945.480217:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 640 00000100:00100000:0.0:1789762945.480242:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067546496:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 45us (186us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762945.480248:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 641 00000100:00100000:1.0:1789762945.528871:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418775834101 time 418775927071 state 0x0 ref 1 00000100:00100000:1.0:1789762945.528913:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e75f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067545856:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762945.529009:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006144512e/0xdf5a7ac59f177819 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.529030:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762945.529035:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762945.529043:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006144512e/0xdf5a7ac59f177819 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.529060:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006144512e/0xdf5a7ac59f177819 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.535905:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006144512e/0xdf5a7ac59f177819 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.535937:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006144512e/0xdf5a7ac59f177819 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.535946:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006144512e/0xdf5a7ac59f177819 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.535957:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c5860c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762945.535962:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006144512e/0xdf5a7ac59f177819 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.535971:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006144512e/0xdf5a7ac59f177819 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.535994:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006144512e/0xdf5a7ac59f177819 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.536004:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006144512e/0xdf5a7ac59f177819 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762945.536071:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e75f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067545856:12345-192.168.203.37@tcp:3:dd.0 Request processed in 7177us (100314us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762945.536084:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1097 00000100:00100000:1.0:1789762945.536096:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418775834101 time 418775927071 state 0x2 ref 0 00000100:00100000:1.0:1789762945.537895:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762945.537963:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067546624 00000100:00100000:3.0:1789762945.537973:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067546624 00000100:00100000:3.0:1789762945.538020:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418775834101 time 418775927071 state 0x0 ref 1 00000100:00100000:3.0:1789762945.538036:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418875927071 time 418775927071 state 0x1 ref 1 00000100:00100000:2.0:1789762945.628988:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418875927071 time 418876043061 state 0x0 ref 1 00000100:00100000:2.0:1789762945.629027:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067546624:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:2.0:1789762945.629083:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000066564079/0xdf5a7ac59f177820 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.629104:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762945.629110:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762945.629118:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000066564079/0xdf5a7ac59f177820 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.629135:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000066564079/0xdf5a7ac59f177820 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.632196:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000066564079/0xdf5a7ac59f177820 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.632211:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000066564079/0xdf5a7ac59f177820 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.632228:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000066564079/0xdf5a7ac59f177820 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.632237:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ff88e000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762945.632242:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000066564079/0xdf5a7ac59f177820 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.632252:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000066564079/0xdf5a7ac59f177820 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.632273:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000066564079/0xdf5a7ac59f177820 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.632283:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000066564079/0xdf5a7ac59f177820 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762945.632356:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067546624:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3345us (94460us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762945.632367:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1098 00000100:00100000:2.0:1789762945.632379:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418875927071 time 418876043061 state 0x2 ref 0 00000100:00100000:0.0:1789762945.634224:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762945.634268:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067546752 00000100:00100000:3.0:1789762945.634278:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067546752 00000100:00100000:3.0:1789762945.634325:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418875927071 time 418876043061 state 0x0 ref 1 00000100:00100000:3.0:1789762945.634338:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418976043061 time 418876043061 state 0x1 ref 1 00010000:00010000:0.0:1789762945.664174:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762945.729075:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418976043061 time 418976129020 state 0x0 ref 1 00000100:00100000:0.0:1789762945.729109:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ee83d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067546752:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:0.0:1789762945.729165:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b1a66ddf/0xdf5a7ac59f177827 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.729187:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762945.729191:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762945.729197:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1a66ddf/0xdf5a7ac59f177827 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.729211:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1a66ddf/0xdf5a7ac59f177827 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.732292:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b1a66ddf/0xdf5a7ac59f177827 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.732306:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b1a66ddf/0xdf5a7ac59f177827 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.732322:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b1a66ddf/0xdf5a7ac59f177827 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.732331:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d98e9200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762945.732338:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1a66ddf/0xdf5a7ac59f177827 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.732348:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b1a66ddf/0xdf5a7ac59f177827 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.732378:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b1a66ddf/0xdf5a7ac59f177827 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.732387:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1a66ddf/0xdf5a7ac59f177827 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762945.732452:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ee83d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067546752:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3357us (98228us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762945.732463:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1099 00000100:00100000:1.0:1789762945.732474:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418976043061 time 418976129020 state 0x2 ref 0 00000100:00100000:1.0:1789762945.734232:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762945.734276:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067546880 00000100:00100000:3.0:1789762945.734286:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067546880 00000100:00100000:3.0:1789762945.734333:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 418976043061 time 418976129020 state 0x0 ref 1 00000100:00100000:3.0:1789762945.734345:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419076129020 time 418976129020 state 0x1 ref 1 00000100:00100000:1.0:1789762945.829190:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419076129020 time 419076250202 state 0x0 ref 1 00000100:00100000:1.0:1789762945.829221:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067546880:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762945.829271:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17782e lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.829289:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762945.829293:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762945.829298:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17782e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.829314:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17782e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.832314:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17782e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.832327:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17782e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.832335:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17782e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.832342:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762945.832345:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17782e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.832352:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17782e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.832368:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17782e lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.832373:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042a8d53d/0xdf5a7ac59f17782e lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762945.832437:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067546880:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3225us (98204us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762945.832450:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1100 00000100:00100000:2.0:1789762945.832460:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419076129020 time 419076250202 state 0x2 ref 0 00000100:00100000:0.0:1789762945.834513:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762945.834565:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067547008 00000100:00100000:2.0:1789762945.834575:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067547008 00000100:00100000:2.0:1789762945.834619:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419076129020 time 419076250202 state 0x0 ref 1 00000100:00100000:2.0:1789762945.834635:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419176250202 time 419076250202 state 0x1 ref 1 00000100:00100000:1.0:1789762945.929335:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419176250202 time 419176395945 state 0x0 ref 1 00000100:00100000:1.0:1789762945.929363:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067547008:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762945.929413:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f177835 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.929432:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762945.929437:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762945.929460:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f177835 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762945.929472:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f177835 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.933819:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f177835 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.933833:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f177835 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.933842:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f177835 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.933848:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ec00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762945.933869:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f177835 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.933880:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f177835 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.933902:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f177835 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.933909:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f177835 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762945.934013:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067547008:12345-192.168.203.37@tcp:3:dd.0 Request processed in 4658us (99499us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762945.934025:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1101 00000100:00100000:3.0:1789762945.934035:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419176250202 time 419176395945 state 0x2 ref 0 00000100:00100000:0.0:1789762945.936823:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762945.936896:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067547136 00000100:00100000:1.0:1789762945.936908:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067547136 00000100:00100000:1.0:1789762945.936988:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419176250202 time 419176395945 state 0x0 ref 1 00000100:00100000:1.0:1789762945.937003:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419276395945 time 419176395945 state 0x1 ref 1 00000100:00100000:1.0:1789762946.029386:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419276395945 time 419276450223 state 0x0 ref 1 00000100:00100000:1.0:1789762946.029410:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067547136:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762946.029449:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17783c lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.029466:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762946.029471:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762946.029476:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17783c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.029489:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17783c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.032446:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17783c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.032457:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17783c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.032464:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17783c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.032471:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762946.032474:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17783c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.032481:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17783c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.032514:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17783c lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.032521:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17783c lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762946.032587:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067547136:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3184us (95764us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762946.032607:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1102 00000100:00100000:1.0:1789762946.032618:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419276395945 time 419276450223 state 0x2 ref 0 00000100:00100000:2.0:1789762946.034750:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762946.034807:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067547264 00000100:00100000:1.0:1789762946.034817:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067547264 00000100:00100000:1.0:1789762946.034855:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419276395945 time 419276450223 state 0x0 ref 1 00000100:00100000:1.0:1789762946.034866:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419376450223 time 419276450223 state 0x1 ref 1 00000100:00100000:3.0:1789762946.129531:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419376450223 time 419376587413 state 0x0 ref 1 00000100:00100000:3.0:1789762946.129561:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067547264:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762946.129628:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177843 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.129646:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762946.129651:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762946.129656:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177843 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.129674:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177843 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.133104:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177843 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.133121:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177843 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.133134:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177843 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.133143:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762946.133147:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177843 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.133163:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177843 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.133184:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177843 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.133191:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177843 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762946.133252:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067547264:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3702us (98502us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762946.133260:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1103 00000100:00100000:3.0:1789762946.133269:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419376450223 time 419376587413 state 0x2 ref 0 00000100:00100000:0.0:1789762946.135052:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762946.135092:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067547392 00000100:00100000:3.0:1789762946.135103:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067547392 00000100:00100000:3.0:1789762946.135150:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419376450223 time 419376587413 state 0x0 ref 1 00000100:00100000:3.0:1789762946.135162:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419476587413 time 419376587413 state 0x1 ref 1 00000100:00100000:3.0:1789762946.229597:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419476587413 time 419476657290 state 0x0 ref 1 00000100:00100000:3.0:1789762946.229630:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067547392:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762946.229679:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17784a lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.229697:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762946.229701:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762946.229707:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17784a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.229721:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17784a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.232772:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17784a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.232783:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17784a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.232789:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17784a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.232794:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762946.232799:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17784a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.232805:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17784a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.232821:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17784a lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.232826:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17784a lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762946.232884:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067547392:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3269us (97833us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762946.232891:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1104 00000100:00100000:3.0:1789762946.232900:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419476587413 time 419476657290 state 0x2 ref 0 00000100:00100000:1.0:1789762946.234857:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762946.234888:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067547520 00000100:00100000:3.0:1789762946.234897:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067547520 00000100:00100000:3.0:1789762946.234955:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419476587413 time 419476657290 state 0x0 ref 1 00000100:00100000:3.0:1789762946.234968:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419576657290 time 419476657290 state 0x1 ref 1 00000100:00100000:2.0:1789762946.329712:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419576657290 time 419576768890 state 0x0 ref 1 00000100:00100000:2.0:1789762946.329743:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067547520:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:2.0:1789762946.329821:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000069c5d7bf/0xdf5a7ac59f177851 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.329847:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762946.329855:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762946.329865:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069c5d7bf/0xdf5a7ac59f177851 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.329884:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069c5d7bf/0xdf5a7ac59f177851 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.333949:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000069c5d7bf/0xdf5a7ac59f177851 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.333965:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069c5d7bf/0xdf5a7ac59f177851 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.333975:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069c5d7bf/0xdf5a7ac59f177851 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.333989:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1c6c800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762946.333995:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069c5d7bf/0xdf5a7ac59f177851 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.334004:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069c5d7bf/0xdf5a7ac59f177851 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.334023:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069c5d7bf/0xdf5a7ac59f177851 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.334030:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069c5d7bf/0xdf5a7ac59f177851 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762946.334101:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067547520:12345-192.168.203.37@tcp:3:dd.0 Request processed in 4367us (99243us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762946.334110:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1105 00000100:00100000:2.0:1789762946.334121:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419576657290 time 419576768890 state 0x2 ref 0 00000100:00100000:2.0:1789762946.336808:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762946.336868:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067547648 00000100:00100000:3.0:1789762946.336881:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067547648 00000100:00100000:3.0:1789762946.336957:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419576657290 time 419576768890 state 0x0 ref 1 00000100:00100000:3.0:1789762946.336972:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419676768890 time 419576768890 state 0x1 ref 1 00000100:00100000:1.0:1789762946.429884:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419676768890 time 419676939886 state 0x0 ref 1 00000100:00100000:1.0:1789762946.429980:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067547648:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762946.430047:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177858 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.430074:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762946.430105:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762946.430114:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177858 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.430133:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177858 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.435596:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177858 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.435637:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177858 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.435658:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177858 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.435675:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762946.435684:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177858 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.435714:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177858 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.435747:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177858 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.435807:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177858 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762946.435914:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067547648:12345-192.168.203.37@tcp:3:dd.0 Request processed in 6002us (99102us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762946.435991:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1106 00000100:00100000:1.0:1789762946.436012:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419676768890 time 419676939886 state 0x2 ref 0 00000100:00100000:2.0:1789762946.439505:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762946.439575:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067547776 00000100:00100000:3.0:1789762946.439589:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067547776 00000100:00100000:3.0:1789762946.439674:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419676768890 time 419676939886 state 0x0 ref 1 00000100:00100000:3.0:1789762946.439695:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419776939886 time 419676939886 state 0x1 ref 1 00000100:00100000:1.0:1789762946.530053:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419776939886 time 419777084279 state 0x0 ref 1 00000100:00100000:1.0:1789762946.530097:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067547776:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762946.530166:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17785f lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.530192:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762946.530199:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762946.530207:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17785f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.530237:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17785f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.535901:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17785f lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.535983:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17785f lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.535999:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17785f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.536012:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762946.536019:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17785f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.536033:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17785f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.536058:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17785f lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.536082:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f17785f lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762946.536196:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067547776:12345-192.168.203.37@tcp:3:dd.0 Request processed in 6111us (96690us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762946.536216:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1107 00000100:00100000:1.0:1789762946.536237:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419776939886 time 419777084279 state 0x2 ref 0 00000100:00100000:0.0:1789762946.540176:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762946.540265:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067547904 00000100:00100000:2.0:1789762946.540280:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067547904 00000100:00100000:2.0:1789762946.540364:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419776939886 time 419777084279 state 0x0 ref 1 00000100:00100000:2.0:1789762946.540382:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419877084279 time 419777084279 state 0x1 ref 1 00000100:00100000:1.0:1789762946.560153:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074746880:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762946.560234:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11d8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074747264:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762946.560255:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762946.560285:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074747008:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762946.560319:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762946.560398:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762946.560406:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074747392:0@lo:400:kworker.0 00000100:00100000:2.0:1789762946.560409:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074747136:0@lo:400:kworker.0 00000100:00100000:0.0:1789762946.560416:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074747776:0@lo:400:kworker.0 00000100:00100000:3.0:1789762946.560458:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074747520:0@lo:400:kworker.0 00000100:00100000:1.0:1789762946.560531:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762946.560542:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762946.560554:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762946.560582:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074746880 00000100:00100000:0.0:1789762946.560593:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074746880 00000100:00100000:2.0:1789762946.560646:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074747648:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762946.560650:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1aed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074747904:0@lo:400:kworker.0 00000100:00100000:3.0:1789762946.560656:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1ac700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074748032:0@lo:400:kworker.0 00000100:00100000:0.0:1789762946.560670:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074747264 00000100:00100000:0.0:1789762946.560677:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074747264 00000100:00100000:0.0:1789762946.560719:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074747008 00000100:00100000:0.0:1789762946.560724:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074747008 00000100:00100000:1.0:1789762946.560789:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762946.560804:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 262 00000100:00100000:2.0:1789762946.560806:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762946.560812:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762946.560836:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074746880:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762946.560863:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1ae300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074748416:0@lo:400:kworker.0 00000100:00100000:2.0:1789762946.560885:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074748160:0@lo:400:kworker.0 00000100:00100000:3.0:1789762946.560888:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1ae680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074748544:0@lo:400:kworker.0 00002000:00000024:0.0:1789762946.560892:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3754950656 avail 3752853504 00000100:00100000:1.0:1789762946.560964:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762946.561044:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762946.561067:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074746880:12345-0@lo:13:osp-pre-0-1.0 Request processed in 229us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762946.561069:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762946.561076:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074746880:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762946.561106:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 262 00000100:00100000:3.0:1789762946.561132:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074747520 00000100:00100000:3.0:1789762946.561140:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074747520 00000100:00100000:2.0:1789762946.561143:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074748672:0@lo:400:kworker.0 00000100:00100000:0.0:1789762946.561154:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074747648 00000100:00100000:1.0:1789762946.561164:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 263 00000100:00100000:0.0:1789762946.561169:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074747648 00000100:00100000:1.0:1789762946.561185:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074747264:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762946.561202:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762946.561222:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074747904 00002000:00000024:1.0:1789762946.561228:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762946.561232:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074747904 00000100:00100000:0.0:1789762946.561242:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 264 00000100:00100000:0.0:1789762946.561261:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074747008:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762946.561286:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762946.561304:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 265 00000100:00100000:3.0:1789762946.561315:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 428 00000100:00100000:1.0:1789762946.561319:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074747264:12345-0@lo:13:osp-pre-1-0.0 Request processed in 137us (997us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762946.561329:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074747648:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762946.561346:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074747520:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762946.561348:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 263 00002000:00000024:2.0:1789762946.561358:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762946.561375:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074747008:12345-0@lo:13:osp-pre-0-0.0 Request processed in 115us (975us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762946.561392:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 264 00000100:00100000:1.0:1789762946.561425:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 429 00000100:00100000:1.0:1789762946.561450:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074747904:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762946.561460:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074747648:12345-0@lo:13:osp-pre-1-1.0 Request processed in 137us (661us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762946.561461:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762946.561472:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074747520:12345-0@lo:400:kworker.0 Request processed in 131us (915us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762946.561478:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762946.561487:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 265 00000100:00100000:3.0:1789762946.561505:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 428 00000100:00100000:0.0:1789762946.561515:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762946.561554:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074747136 00000100:00100000:1.0:1789762946.561559:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074747904:12345-0@lo:400:kworker.0 Request processed in 112us (768us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762946.561565:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074747136 00000100:00100000:0.0:1789762946.561578:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1adf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074748288:0@lo:400:kworker.0 00000100:00100000:3.0:1789762946.561590:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074747392 00000100:00100000:1.0:1789762946.561591:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 429 00000100:00100000:3.0:1789762946.561602:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074747392 00000100:00100000:0.0:1789762946.561639:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762946.561648:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:2.0:1789762946.561676:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074747136:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762946.561691:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074748032 00000100:00100000:3.0:1789762946.561692:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074748416 00000100:00100000:3.0:1789762946.561706:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074748416 00000100:00100000:0.0:1789762946.561709:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11d8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074747264:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762946.561709:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074748032 00000100:00100000:3.0:1789762946.561797:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074748160 00000100:00100000:0.0:1789762946.561808:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074747776 00000100:00100000:3.0:1789762946.561809:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074748160 00000100:00100000:2.0:1789762946.561810:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074747136:12345-0@lo:400:kworker.0 Request processed in 137us (1263us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762946.561812:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074748288 00000100:00100000:0.0:1789762946.561823:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074747776 00000100:00100000:1.0:1789762946.561826:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074748288 00000100:00100000:2.0:1789762946.561838:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00100000:3.0:1789762946.561888:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 642 00000100:00100000:0.0:1789762946.561902:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 643 00000100:00100000:3.0:1789762946.561912:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074747392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762946.561963:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074748544 00000100:00100000:1.0:1789762946.561973:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 644 00000100:00100000:2.0:1789762946.561980:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074748544 00000100:00100000:0.0:1789762946.561982:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074748416:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762946.562000:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074748032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762946.562024:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074747392:12345-0@lo:400:kworker.0 Request processed in 119us (1495us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762946.562036:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 642 00000100:00100000:2.0:1789762946.562068:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074748672 00000100:00100000:2.0:1789762946.562080:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074748672 00000100:00100000:3.0:1789762946.562081:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 645 00000100:00100000:0.0:1789762946.562102:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074748416:12345-0@lo:400:kworker.0 Request processed in 177us (1177us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762946.562104:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074748032:12345-0@lo:400:kworker.0 Request processed in 112us (1306us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762946.562113:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074748160:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762946.562141:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 262 00000100:00100000:1.0:1789762946.562144:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 644 00000100:00100000:2.0:1789762946.562169:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074748544:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762946.562170:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 643 00000100:00100000:1.0:1789762946.562188:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 646 00000100:00100000:1.0:1789762946.562212:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074747776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762946.562218:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074748160:12345-0@lo:400:kworker.0 Request processed in 111us (1155us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762946.562230:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 647 00000100:00100000:3.0:1789762946.562245:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 645 00000100:00100000:0.0:1789762946.562256:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074748288:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762946.562272:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074748544:12345-0@lo:400:kworker.0 Request processed in 114us (1230us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762946.562284:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 262 00000100:00100000:3.0:1789762946.562312:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 263 00000100:00100000:1.0:1789762946.562323:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074747776:12345-0@lo:400:kworker.0 Request processed in 117us (812us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762946.562345:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074748672:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762946.562347:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 646 00000100:00100000:0.0:1789762946.562355:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074748288:12345-0@lo:400:kworker.0 Request processed in 103us (714us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762946.562369:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074747136:0@lo:400:kworker.0 00000100:00100000:0.0:1789762946.562390:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 647 00000100:00100000:2.0:1789762946.562417:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074747648:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762946.562452:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074748672:12345-0@lo:400:kworker.0 Request processed in 117us (1246us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762946.562454:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762946.562455:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074748160:0@lo:400:kworker.0 00000100:00080000:0.0:1789762946.562470:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762946.562474:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074747392:0@lo:400:kworker.0 00000100:00100000:3.0:1789762946.562498:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 263 00000100:00100000:2.0:1789762946.562504:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074748672:0@lo:400:kworker.0 00000100:00100000:1.0:1789762946.562521:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1aed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074747904:0@lo:400:kworker.0 00000100:00100000:0.0:1789762946.562544:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1ac700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074748032:0@lo:400:kworker.0 00000100:00100000:1.0:1789762946.562574:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1ae300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074748416:0@lo:400:kworker.0 00000100:00100000:0.0:1789762946.562587:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1ae680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074748544:0@lo:400:kworker.0 00000100:00100000:0.0:1789762946.562633:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074747008:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762946.562649:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074747776:0@lo:400:kworker.0 00000100:00100000:0.0:1789762946.562667:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074747520:0@lo:400:kworker.0 00000100:00100000:2.0:1789762946.562693:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1adf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074748288:0@lo:400:kworker.0 00000100:00100000:1.0:1789762946.630167:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419877084279 time 419877218822 state 0x0 ref 1 00000100:00100000:1.0:1789762946.630206:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067547904:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762946.630271:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177866 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.630297:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762946.630304:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762946.630313:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177866 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.630334:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177866 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.635842:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177866 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.635865:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177866 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.635877:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177866 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.635887:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998e200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762946.635893:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177866 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.635905:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177866 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.635958:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177866 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.635969:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000809719ad/0xdf5a7ac59f177866 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762946.636059:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067547904:12345-192.168.203.37@tcp:3:dd.0 Request processed in 5865us (95884us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762946.636074:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1108 00000100:00100000:3.0:1789762946.636094:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419877084279 time 419877218822 state 0x2 ref 0 00000100:00100000:0.0:1789762946.639484:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762946.639575:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067548032 00000100:00100000:1.0:1789762946.639591:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067548032 00000100:00100000:1.0:1789762946.639660:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419877084279 time 419877218822 state 0x0 ref 1 00000100:00100000:1.0:1789762946.639680:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419977218822 time 419877218822 state 0x1 ref 1 00010000:00010000:1.0:1789762946.688143:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762946.730281:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419977218822 time 419977326497 state 0x0 ref 1 00000100:00100000:1.0:1789762946.730335:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067548032:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762946.730406:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f17786d lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.730433:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762946.730440:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762946.730449:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f17786d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.730478:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f17786d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.735011:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f17786d lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.735038:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f17786d lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.735060:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f17786d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.735079:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998f800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762946.735090:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f17786d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.735112:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f17786d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.735141:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f17786d lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.735151:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f14038f9/0xdf5a7ac59f17786d lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762946.735257:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067548032:12345-192.168.203.37@tcp:3:dd.0 Request processed in 4941us (95774us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762946.735277:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1109 00000100:00100000:1.0:1789762946.735296:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419977218822 time 419977326497 state 0x2 ref 0 00000100:00100000:0.0:1789762946.738456:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762946.738556:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067548160 00000100:00100000:1.0:1789762946.738572:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067548160 00000100:00100000:1.0:1789762946.738685:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 419977218822 time 419977326497 state 0x0 ref 1 00000100:00100000:1.0:1789762946.738715:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420077326497 time 419977326497 state 0x1 ref 1 00000100:00100000:3.0:1789762946.830363:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420077326497 time 420077421343 state 0x0 ref 1 00000100:00100000:3.0:1789762946.830397:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067548160:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762946.830448:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177874 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.830466:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762946.830470:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762946.830475:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177874 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.830491:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177874 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.833443:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177874 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.833460:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177874 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.833471:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177874 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.833487:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762946.833493:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177874 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.833504:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177874 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.833526:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177874 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.833535:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177874 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762946.833601:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067548160:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3217us (95148us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762946.833613:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1110 00000100:00100000:3.0:1789762946.833625:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420077326497 time 420077421343 state 0x2 ref 0 00000100:00100000:0.0:1789762946.835685:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762946.835730:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067548288 00000100:00100000:3.0:1789762946.835741:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067548288 00000100:00100000:3.0:1789762946.835871:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420077326497 time 420077421343 state 0x0 ref 1 00000100:00100000:3.0:1789762946.835886:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420177421343 time 420077421343 state 0x1 ref 1 00000100:00100000:3.0:1789762946.930446:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420177421343 time 420177510422 state 0x0 ref 1 00000100:00100000:3.0:1789762946.930474:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067548288:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762946.930511:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17787b lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.930528:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762946.930534:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762946.930541:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17787b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.930557:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17787b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.933473:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17787b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.933484:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17787b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.933491:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17787b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.933496:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762946.933500:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17787b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.933506:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17787b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.933525:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17787b lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.933531:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f17787b lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762946.933587:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067548288:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3126us (97903us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762946.933595:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1111 00000100:00100000:3.0:1789762946.933602:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420177421343 time 420177510422 state 0x2 ref 0 00000100:00100000:2.0:1789762946.935361:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762946.935424:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067548416 00000100:00100000:1.0:1789762946.935438:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067548416 00000100:00100000:1.0:1789762946.935501:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420177421343 time 420177510422 state 0x0 ref 1 00000100:00100000:1.0:1789762946.935515:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420277510422 time 420177510422 state 0x1 ref 1 00000100:00100000:2.0:1789762947.030535:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420277510422 time 420277595522 state 0x0 ref 1 00000100:00100000:2.0:1789762947.030577:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067548416:12345-192.168.203.37@tcp:3:dd.0 00000100:00080000:0.0:1789762947.030605:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789762947.030626:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177882 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.030643:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762947.030647:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762947.030653:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177882 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.030670:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177882 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.033659:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177882 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.033670:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177882 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.033677:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177882 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.033682:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa0400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762947.033686:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177882 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.033693:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177882 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.033708:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177882 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.033713:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177882 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762947.033787:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067548416:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3235us (98431us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762947.033795:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1112 00000100:00100000:2.0:1789762947.033803:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420277510422 time 420277595522 state 0x2 ref 0 00000100:00100000:2.0:1789762947.035599:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762947.035660:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067548544 00000100:00100000:3.0:1789762947.035673:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067548544 00000100:00100000:3.0:1789762947.035726:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420277510422 time 420277595522 state 0x0 ref 1 00000100:00100000:3.0:1789762947.035738:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420377595522 time 420277595522 state 0x1 ref 1 00000100:00100000:3.0:1789762947.130591:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420377595522 time 420377657065 state 0x0 ref 1 00000100:00100000:3.0:1789762947.130617:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067548544:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762947.130654:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177889 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.130667:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762947.130680:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762947.130685:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177889 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.130696:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177889 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.133530:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177889 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.133541:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177889 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.133549:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177889 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.133555:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762947.133559:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177889 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.133565:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177889 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.133579:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177889 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.133585:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f177889 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762947.133639:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067548544:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3032us (98041us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762947.133646:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1113 00000100:00100000:3.0:1789762947.133652:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420377595522 time 420377657065 state 0x2 ref 0 00000100:00100000:0.0:1789762947.135717:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762947.135767:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067548672 00000100:00100000:3.0:1789762947.135774:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067548672 00000100:00100000:3.0:1789762947.135809:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420377595522 time 420377657065 state 0x0 ref 1 00000100:00100000:3.0:1789762947.135820:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420477657065 time 420377657065 state 0x1 ref 1 00000100:00100000:2.0:1789762947.230663:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420477657065 time 420477729377 state 0x0 ref 1 00000100:00100000:2.0:1789762947.230686:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067548672:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:2.0:1789762947.230722:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177890 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.230736:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762947.230739:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762947.230743:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177890 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.230770:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177890 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.233692:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177890 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.233700:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177890 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.233706:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177890 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.233711:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d1fa0400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762947.233714:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177890 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.233720:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177890 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.233733:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177890 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.233738:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afb2511f/0xdf5a7ac59f177890 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762947.233806:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067548672:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3127us (98094us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762947.233815:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1114 00000100:00100000:2.0:1789762947.233822:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420477657065 time 420477729377 state 0x2 ref 0 00000100:00100000:0.0:1789762947.235740:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762947.235792:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067548800 00000100:00100000:2.0:1789762947.235803:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067548800 00000100:00100000:2.0:1789762947.235838:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420477657065 time 420477729377 state 0x0 ref 1 00000100:00100000:2.0:1789762947.235851:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420577729377 time 420477729377 state 0x1 ref 1 00000100:00100000:1.0:1789762947.330780:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420577729377 time 420577825859 state 0x0 ref 1 00000100:00100000:1.0:1789762947.330813:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067548800:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762947.330863:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001932250a/0xdf5a7ac59f177897 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.330881:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762947.330886:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762947.330892:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001932250a/0xdf5a7ac59f177897 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.330906:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001932250a/0xdf5a7ac59f177897 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.333814:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001932250a/0xdf5a7ac59f177897 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.333828:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001932250a/0xdf5a7ac59f177897 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.333836:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001932250a/0xdf5a7ac59f177897 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.333843:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c21a4600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762947.333849:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001932250a/0xdf5a7ac59f177897 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.333856:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001932250a/0xdf5a7ac59f177897 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.333877:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001932250a/0xdf5a7ac59f177897 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.333882:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001932250a/0xdf5a7ac59f177897 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762947.333980:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067548800:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3178us (98244us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762947.333988:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1115 00000100:00100000:0.0:1789762947.333999:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420577729377 time 420577825859 state 0x2 ref 0 00000100:00100000:0.0:1789762947.335709:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762947.335773:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067548928 00000100:00100000:3.0:1789762947.335784:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067548928 00000100:00100000:3.0:1789762947.335835:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420577729377 time 420577825859 state 0x0 ref 1 00000100:00100000:3.0:1789762947.335847:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420677825859 time 420577825859 state 0x1 ref 1 00000100:00100000:1.0:1789762947.430877:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420677825859 time 420677942442 state 0x0 ref 1 00000100:00100000:1.0:1789762947.430901:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067548928:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762947.430961:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005f463122/0xdf5a7ac59f17789e lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.430976:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762947.430979:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762947.430984:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005f463122/0xdf5a7ac59f17789e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.430995:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005f463122/0xdf5a7ac59f17789e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.434019:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005f463122/0xdf5a7ac59f17789e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.434029:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005f463122/0xdf5a7ac59f17789e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.434036:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005f463122/0xdf5a7ac59f17789e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.434045:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36c21a5600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762947.434049:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005f463122/0xdf5a7ac59f17789e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.434055:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005f463122/0xdf5a7ac59f17789e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.434068:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005f463122/0xdf5a7ac59f17789e lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.434072:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005f463122/0xdf5a7ac59f17789e lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762947.434120:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067548928:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3228us (98412us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762947.434128:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1116 00000100:00100000:3.0:1789762947.434135:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420677825859 time 420677942442 state 0x2 ref 0 00000100:00100000:0.0:1789762947.435935:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762947.435983:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067549056 00000100:00100000:2.0:1789762947.435992:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067549056 00000100:00100000:2.0:1789762947.436028:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420677825859 time 420677942442 state 0x0 ref 1 00000100:00100000:2.0:1789762947.436038:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420777942442 time 420677942442 state 0x1 ref 1 00000100:00100000:3.0:1789762947.530990:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420777942442 time 420778047960 state 0x0 ref 1 00000100:00100000:3.0:1789762947.531025:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067549056:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762947.531076:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1778a5 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.531100:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762947.531105:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762947.531110:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1778a5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.531124:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1778a5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.534063:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1778a5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.534076:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1778a5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.534083:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1778a5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.534089:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762947.534093:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1778a5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.534101:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1778a5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.534120:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1778a5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.534130:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b6c16a4/0xdf5a7ac59f1778a5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762947.534186:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2396:x1876702067549056:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3178us (98273us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762947.534194:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1117 00000100:00100000:2.0:1789762947.534202:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420777942442 time 420778047960 state 0x2 ref 0 00000100:00100000:0.0:1789762947.536190:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762947.536240:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067549184 00000100:00100000:1.0:1789762947.536250:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067549184 00000100:00100000:1.0:1789762947.536302:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420777942442 time 420778047960 state 0x0 ref 1 00000100:00100000:1.0:1789762947.536314:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420878047960 time 420778047960 state 0x1 ref 1 00000100:00100000:1.0:1789762947.631092:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420878047960 time 420878152504 state 0x0 ref 1 00000100:00100000:1.0:1789762947.631127:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067549184:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762947.631170:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778ac lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.631189:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762947.631196:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762947.631204:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778ac lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.631223:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778ac lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.634488:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778ac lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.634498:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778ac lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.634504:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778ac lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.634509:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ec00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762947.634513:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778ac lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.634520:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778ac lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.634533:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778ac lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.634538:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778ac lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762947.634586:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067549184:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3476us (98403us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762947.634593:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1118 00000100:00100000:1.0:1789762947.634600:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420878047960 time 420878152504 state 0x2 ref 0 00000100:00100000:1.0:1789762947.636335:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762947.636388:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067549312 00000100:00100000:3.0:1789762947.636402:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067549312 00000100:00100000:3.0:1789762947.636461:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420878047960 time 420878152504 state 0x0 ref 1 00000100:00100000:3.0:1789762947.636474:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420978152504 time 420878152504 state 0x1 ref 1 00010000:00010000:1.0:1789762947.712148:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762947.731209:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420978152504 time 420978271530 state 0x0 ref 1 00000100:00100000:1.0:1789762947.731238:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79ac850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067549312:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762947.731288:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778b3 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.731305:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762947.731309:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762947.731315:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778b3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.731328:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778b3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.734320:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778b3 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.734332:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778b3 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.734341:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778b3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.734346:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d998ec00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762947.734350:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778b3 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.734357:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778b3 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.734374:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778b3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.734379:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032c78cf5/0xdf5a7ac59f1778b3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762947.734437:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79ac850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067549312:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3211us (98103us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762947.734446:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1119 00000100:00100000:1.0:1789762947.734455:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420978152504 time 420978271530 state 0x2 ref 0 00000100:00100000:1.0:1789762947.736373:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762947.736414:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067549440 00000100:00100000:2.0:1789762947.736427:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067549440 00000100:00100000:2.0:1789762947.736482:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 420978152504 time 420978271530 state 0x0 ref 1 00000100:00100000:2.0:1789762947.736495:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421078271530 time 420978271530 state 0x1 ref 1 00000100:00100000:3.0:1789762947.831303:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421078271530 time 421078357670 state 0x0 ref 1 00000100:00100000:3.0:1789762947.831337:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067549440:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762947.831386:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778ba lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.831408:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762947.831414:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762947.831422:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778ba lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.831449:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778ba lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.834660:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778ba lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.834674:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778ba lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.834685:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778ba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.834694:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caec00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762947.834700:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778ba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.834713:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778ba lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.834732:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778ba lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762947.834741:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778ba lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762947.834852:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067549440:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3528us (98477us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762947.834863:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1120 00000100:00100000:3.0:1789762947.834874:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421078271530 time 421078357670 state 0x2 ref 0 00000100:00100000:0.0:1789762947.836905:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762947.836958:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067549568 00000100:00100000:1.0:1789762947.836969:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067549568 00000100:00100000:1.0:1789762947.837014:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421078271530 time 421078357670 state 0x0 ref 1 00000100:00100000:1.0:1789762947.837027:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421178357670 time 421078357670 state 0x1 ref 1 00000100:00100000:2.0:1789762947.931404:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421178357670 time 421178465792 state 0x0 ref 1 00000100:00100000:2.0:1789762947.931435:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067549568:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:2.0:1789762947.931501:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1778c1 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.931520:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762947.931524:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762947.931530:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1778c1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.931547:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1778c1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.935119:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1778c1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.935131:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1778c1 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.935140:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1778c1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.935149:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762947.935155:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1778c1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.935167:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1778c1 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.935190:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1778c1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.935199:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa815244/0xdf5a7ac59f1778c1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762947.935275:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067549568:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3852us (98368us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762947.935288:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1121 00000100:00100000:2.0:1789762947.935297:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421178357670 time 421178465792 state 0x2 ref 0 00000100:00100000:2.0:1789762947.937066:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762947.937110:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067549696 00000100:00100000:1.0:1789762947.937120:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067549696 00000100:00100000:1.0:1789762947.937164:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421178357670 time 421178465792 state 0x0 ref 1 00000100:00100000:1.0:1789762947.937176:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421278465792 time 421178465792 state 0x1 ref 1 00000100:00100000:1.0:1789762948.031496:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421278465792 time 421278559869 state 0x0 ref 1 00000100:00100000:1.0:1789762948.031524:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067549696:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:1.0:1789762948.031567:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e4f42567/0xdf5a7ac59f1778c8 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.031591:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762948.031596:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762948.031602:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4f42567/0xdf5a7ac59f1778c8 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.031632:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4f42567/0xdf5a7ac59f1778c8 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.035137:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e4f42567/0xdf5a7ac59f1778c8 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.035150:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4f42567/0xdf5a7ac59f1778c8 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.035158:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4f42567/0xdf5a7ac59f1778c8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.035164:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ca3a7a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762948.035168:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4f42567/0xdf5a7ac59f1778c8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.035174:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4f42567/0xdf5a7ac59f1778c8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.035191:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4f42567/0xdf5a7ac59f1778c8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.035196:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4f42567/0xdf5a7ac59f1778c8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762948.035257:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067549696:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3743us (98191us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762948.035265:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1122 00000100:00100000:0.0:1789762948.035274:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421278465792 time 421278559869 state 0x2 ref 0 00000100:00100000:2.0:1789762948.037085:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762948.037128:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067549824 00000100:00100000:3.0:1789762948.037138:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067549824 00000100:00100000:3.0:1789762948.037183:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421278465792 time 421278559869 state 0x0 ref 1 00000100:00100000:3.0:1789762948.037194:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421378559869 time 421278559869 state 0x1 ref 1 00000100:00100000:3.0:1789762948.131563:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421378559869 time 421378622689 state 0x0 ref 1 00000100:00100000:3.0:1789762948.131598:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067549824:12345-192.168.203.37@tcp:3:dd.0 00010000:00010000:3.0:1789762948.131647:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778cf lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.131665:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762948.131670:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762948.131675:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778cf lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.131713:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778cf lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.134689:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778cf lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.134698:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778cf lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.134705:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778cf lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.134710:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caec00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762948.134715:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778cf lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.134722:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778cf lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.134739:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778cf lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.134758:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081d34d09/0xdf5a7ac59f1778cf lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762948.134824:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2395:x1876702067549824:12345-192.168.203.37@tcp:3:dd.0 Request processed in 3235us (97740us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762948.134831:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1123 00000100:00100000:3.0:1789762948.134840:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421378559869 time 421378622689 state 0x2 ref 0 00000100:00100000:1.0:1789762948.164147:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762948.164208:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067549952 00000100:00100000:2.0:1789762948.164221:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067549952 00000100:00100000:2.0:1789762948.164297:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 72 00000100:00100000:2.0:1789762948.164308:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:6357:x1876702067549952:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762948.164325:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762948.164378:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ed446dc0/0xdf5a7ac59f177613 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cad2 expref: 6 pid: 9038 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.164406:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762948.164461:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:6357:x1876702067549952:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 156us (327us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762948.164467:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 72 00000100:00100000:1.0:1789762948.172965:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762948.173011:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067550080 00000100:00100000:3.0:1789762948.173017:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067550080 00000100:00100000:3.0:1789762948.173062:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 354 00000100:00100000:3.0:1789762948.173073:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:13087:x1876702067550080:12345-192.168.203.37@tcp:101:bash.0 00000100:00080000:0.0:1789762948.173080:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:3.0:1789762948.173092:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762948.173113:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caec00/0xdf5a7ac59f1778d6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cad9 expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.173178:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1778dd lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.173191:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1778dd lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.173201:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1778dd lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762948.173234:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17:0x0]: returning size 16384 00000004:00200000:3.0:1789762948.173237:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762948.173254:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1778dd lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.173280:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caec00/0xdf5a7ac59f1778d6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cad9 expref: 10 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.173302:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1778dd lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cad9 expref: 9 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.173316:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:3.0:1789762948.173319:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789762948.173322:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8cafc00, rc 0) 00000100:00100000:3.0:1789762948.173356:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:13087:x1876702067550080:12345-192.168.203.37@tcp:101:bash.0 Request processed in 288us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762948.173361:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 354 00000100:00100000:0.0:1789762948.177262:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762948.177321:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067550336 00000100:00100000:1.0:1789762948.177327:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067550336 00000100:00100000:1.0:1789762948.177361:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 355 00000100:00100000:1.0:1789762948.177368:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:15723:x1876702067550336:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:1.0:1789762948.177381:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762948.177393:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f1778e4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cae0 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762948.177445:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17:0x0]: returning size 16384 00010000:00010000:1.0:1789762948.177618:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f1778e4 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cae0 expref: 11 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.177627:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:1.0:1789762948.177629:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789762948.177632:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f1778e4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cae0 expref: 11 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.177647:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee81b000, rc 0) 00000100:00100000:1.0:1789762948.177675:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:15723:x1876702067550336:12345-192.168.203.37@tcp:101:rm.0 Request processed in 309us (416us total) trans 4294967459 rc 301/301 00000100:00100000:1.0:1789762948.177682:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 355 00000100:00100000:1.0:1789762948.179070:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762948.179124:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067550464 00000100:00100000:0.0:1789762948.179131:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067550464 00000100:00100000:0.0:1789762948.179175:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 103 00000100:00100000:0.0:1789762948.179186:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:15723:x1876702067550464:12345-192.168.203.37@tcp:37:rm.0 00000100:00100000:0.0:1789762948.181374:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:15723:x1876702067550464:12345-192.168.203.37@tcp:37:rm.0 Request processed in 2191us (2303us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762948.181380:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 103 00000100:00100000:2.0:1789762948.182623:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762948.182678:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067550720 00000100:00100000:0.0:1789762948.182684:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067550720 00000100:00100000:0.0:1789762948.182718:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 104 00000100:00100000:0.0:1789762948.182730:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:15723:x1876702067550720:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:0.0:1789762948.182887:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15723:x1876702067550720:12345-192.168.203.37@tcp:35:rm.0 Request processed in 159us (265us total) trans 4294967460 rc 0/0 00000100:00100000:0.0:1789762948.182897:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 104 00000100:00100000:1.0:1789762948.183736:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762948.183782:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067550976 00000100:00100000:0.0:1789762948.183787:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067550976 00000100:00100000:0.0:1789762948.183812:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 356 00000100:00100000:0.0:1789762948.183818:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:15723:x1876702067550976:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:0.0:1789762948.183827:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762948.183838:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1778f2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cae7 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.183875:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1778f9 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.183884:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1778f9 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x17:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.183890:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1778f9 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x17:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762948.183896:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17:0x0]: returning size 16384 00010000:00010000:0.0:1789762948.183957:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1778f2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cae7 expref: 14 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.183969:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1778f9 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x17:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cae7 expref: 13 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.183975:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:0.0:1789762948.183977:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1789762948.183979:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d14f0400, rc 0) 00000100:00100000:0.0:1789762948.184003:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:15723:x1876702067550976:12345-192.168.203.37@tcp:101:rm.0 Request processed in 186us (269us total) trans 4294967461 rc 0/0 00000100:00100000:0.0:1789762948.184008:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 356 00000100:00100000:2.0:1789762948.184951:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762948.184996:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067551104 00000100:00100000:3.0:1789762948.185001:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067551104 00000100:00080000:0.0:1789762948.185033:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762948.185039:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 357 00000100:00100000:3.0:1789762948.185048:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1aca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15723:x1876702067551104:12345-192.168.203.37@tcp:41:rm.0 00000004:00000024:3.0:1789762948.185099:0:7744:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7614758912 avail 7610564608 00000100:00100000:3.0:1789762948.185134:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1aca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:15723:x1876702067551104:12345-192.168.203.37@tcp:41:rm.0 Request processed in 88us (185us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762948.185140:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 357 00000100:00100000:2.0:1789762948.185883:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762948.185941:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067551360 00000100:00100000:0.0:1789762948.185947:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067551360 00000100:00100000:0.0:1789762948.185979:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 105 00000100:00100000:0.0:1789762948.185989:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:15723:x1876702067551360:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:0.0:1789762948.186207:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15723:x1876702067551360:12345-192.168.203.37@tcp:35:rm.0 Request processed in 219us (322us total) trans 4294967462 rc 0/0 00000100:00100000:0.0:1789762948.186217:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 105 00000100:00100000:1.0:1789762948.186865:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762948.186913:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067551616 00000100:00100000:2.0:1789762948.186939:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067551616 00000100:00100000:2.0:1789762948.186971:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 358 00000100:00100000:2.0:1789762948.186979:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15723:x1876702067551616:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:2.0:1789762948.187005:0:13389:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789762948.187019:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ca3a7e00/0xdf5a7ac59f177605 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cacb expref: 14 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.187034:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae200/0xdf5a7ac59f177334 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cac4 expref: 13 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.187042:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x18:0x0].0x0 (ffff9c37d3beeb00) 00010000:00010000:2.0:1789762948.187050:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cafc00/0xdf5a7ac59f1778dd lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cad9 expref: 12 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.187057:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762948.187059:0:13389:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762948.187069:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177907 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.187076:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177907 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.187082:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177907 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.187091:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f17790e lrc: 3/0,1 mode: --/PW res: [0x240000402:0x17:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.187096:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f17790e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.187100:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f17790e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.187366:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177915 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.187373:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177915 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.187379:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177915 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789762948.188320:0:13389:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].14, 64 off9088 00000004:00080000:2.0:1789762948.188329:0:13389:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 421435409929ns, next at 422435407932ns, rc = 0 00010000:00010000:2.0:1789762948.188364:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17791c lrc: 3/0,1 mode: --/PW res: [0x240000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188371:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17791c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188377:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17791c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188383:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17791c lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188389:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17791c lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188393:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17791c lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188410:0:13389:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762948.188421:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177915 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188426:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177915 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188431:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177915 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188435:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762948.188438:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177915 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188444:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177915 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188450:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x18:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762948.188452:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762948.188455:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177915 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188460:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177915 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x18:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.188462:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762948.188471:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17791c lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188475:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177907 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.188480:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762948.188484:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177907 lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.188486:0:7217:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17791c lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188499:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177907 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.188502:0:7217:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17791c lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188511:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177907 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.188517:0:7217:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x18:0x0].0x0 (ffff9c37d3beed00) 00010000:00010000:2.0:1789762948.188518:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762948.188524:0:7217:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789762948.188524:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177907 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.188529:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17791c lrc: 1/0,0 mode: --/PW res: [0x240000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188532:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177907 lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.188537:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17791c lrc: 0/0,0 mode: --/PW res: [0x240000402:0x18:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188542:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:2.0:1789762948.188546:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762948.188549:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177907 lrc: 3/0,0 mode: --/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.188553:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762948.188554:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177907 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.188557:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762948.188560:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00010000:2.0:1789762948.188567:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000021556567 reply state 00000000fafe3db8 transno 4294967463 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f17790e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188579:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f17790e lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.188586:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x4009f790 (ffff9c37d3bee100) 00010000:00010000:2.0:1789762948.188589:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762948.188591:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762948.188636:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:15723:x1876702067551616:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1656us (1770us total) trans 4294967463 rc 0/0 00000100:00100000:2.0:1789762948.188643:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 358 00000100:00100000:2.0:1789762948.189617:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762948.189658:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762948.189663:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067551744 00000100:00100000:1.0:1789762948.189669:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067551744 00000100:00100000:0.0:1789762948.189699:0:13186:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067552000 00000100:00100000:0.0:1789762948.189706:0:13186:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067552000 00000100:00100000:1.0:1789762948.189706:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 73 00000100:00100000:1.0:1789762948.189719:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ac380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067551744:12345-192.168.203.37@tcp:103:rm.0 00010000:00010000:1.0:1789762948.189729:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762948.189756:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6000/0xdf5a7ac59f177293 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca8c expref: 7 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762948.189763:0:13186:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 106 00010000:00010000:1.0:1789762948.189779:0:6404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37e37a7800) 00000100:00100000:0.0:1789762948.189780:0:13186:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbfc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:15723:x1876702067552000:12345-192.168.203.37@tcp:35:rm.0 00010000:00010000:1.0:1789762948.189791:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762948.189822:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ac380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2398:x1876702067551744:12345-192.168.203.37@tcp:103:rm.0 Request processed in 106us (207us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762948.189828:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 73 00000100:00100000:0.0:1789762948.189901:0:13186:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbfc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15723:x1876702067552000:12345-192.168.203.37@tcp:35:rm.0 Request processed in 125us (241us total) trans 4294967464 rc 0/0 00000100:00100000:0.0:1789762948.189911:0:13186:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 106 00000100:00100000:2.0:1789762948.190491:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762948.190533:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067552256 00000100:00100000:3.0:1789762948.190540:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067552256 00000100:00100000:3.0:1789762948.190570:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 359 00000100:00100000:3.0:1789762948.190575:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:15723:x1876702067552256:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:3.0:1789762948.190597:0:6420:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1789762948.190607:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1778f9 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x17:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cae7 expref: 14 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.190620:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177318 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cabd expref: 13 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.190631:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37d3bee800) 00010000:00010000:3.0:1789762948.190636:0:6420:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762948.190670:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.190677:0:6420:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.190683:0:6420:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762948.190701:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb0000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074748800:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1789762948.190735:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762948.190784:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074748800 00000100:00100000:0.0:1789762948.190790:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074748800 00000100:00100000:3.0:1789762948.190792:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c96ab9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789762948.190831:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 430 00000100:00100000:0.0:1789762948.190842:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074748800:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1789762948.190853:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762948.190866:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f17792a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f177923 expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.190888:0:6424:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6600/0xdf5a7ac59f1772a8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca9a expref: 5 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.190900:0:6424:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f17792a lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f177923 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.190960:0:6424:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6600/0xdf5a7ac59f1772a8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca9a expref: 5 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.190977:0:6424:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6600/0xdf5a7ac59f1772a8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca9a expref: 5 pid: 13389 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762948.190990:0:6424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11da680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074748928:192.168.203.37@tcp:104: 00000100:00100000:0.0:1789762948.191044:0:6424:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d27240c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789762948.191764:0:6424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11da680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074748928:192.168.203.37@tcp:104: 00010000:00010000:0.0:1789762948.191786:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f17792a lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177923 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.191797:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762948.191800:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762948.191805:0:6424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f17792a lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177923 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.191815:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d14f1a00, rc 0) 00000100:00100000:2.0:1789762948.191847:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762948.191862:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6420:x1876702074748800:12345-0@lo:101:mdt00_002.0 Request processed in 1020us (1122us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762948.191871:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 430 00000100:00100000:1.0:1789762948.191893:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067552384 00000100:00100000:1.0:1789762948.191898:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067552384 00010000:00010000:1.0:1789762948.191934:0:8070:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6600/0xdf5a7ac59f1772a8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca9a expref: 6 pid: 13389 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762948.191942:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb0000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074748800:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1789762948.191961:0:6420:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762948.191970:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 90 00010000:00010000:3.0:1789762948.191972:0:6420:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762948.191979:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1adf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067552384:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762948.191988:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762948.191995:0:8070:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6600/0xdf5a7ac59f1772a8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca9a expref: 6 pid: 13389 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.192008:0:8070:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6600/0xdf5a7ac59f1772a8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca9a expref: 6 pid: 13389 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.192023:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ca3a6600/0xdf5a7ac59f1772a8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9ca9a expref: 6 pid: 13389 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.192035:0:8070:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762948.192040:0:8070:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f17792a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177923 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.192052:0:8070:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f17792a lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f177923 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.192061:0:8070:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f17792a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f177923 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.192085:0:8070:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f17792a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177923 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762948.192095:0:8070:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:8070:1876702074749056:0@lo:105: 00000100:00100000:1.0:1789762948.192119:0:8070:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762948.192147:0:8070:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c9c70f00 going to sleep for 11 seconds 00000100:00100000:3.0:1789762948.192153:0:6402:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074749056 00000100:00100000:3.0:1789762948.192157:0:6402:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074749056 00000100:00100000:3.0:1789762948.192187:0:6402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 59 00000100:00100000:3.0:1789762948.192194:0:6402:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:8070:x1876702074749056:12345-0@lo:105: 00010000:00010000:3.0:1789762948.192203:0:6402:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.192214:0:6402:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.192221:0:6402:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.192226:0:6402:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.192232:0:6402:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.192237:0:6402:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.192252:0:6402:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c37d8caec00) 00010000:00010000:0.0:1789762948.192275:0:6420:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.192285:0:6420:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762948.192287:0:6402:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:8070:x1876702074749056:12345-0@lo:105: Request processed in 91us (164us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762948.192302:0:6402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 59 00000004:00080000:0.0:1789762948.192306:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1789762948.192323:0:8070:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:8070:1876702074749056:0@lo:105: 00010000:00010000:0.0:1789762948.192328:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d14f1400/0xdf5a7ac59f177931 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.192335:0:6420:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d14f1400/0xdf5a7ac59f177931 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.192338:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762948.192341:0:6420:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d14f1400/0xdf5a7ac59f177931 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762948.192354:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11da680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074749184:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1789762948.192379:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1adf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067552384:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 393us (524us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762948.192382:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762948.192393:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 90 00000100:00100000:0.0:1789762948.192406:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d27240c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762948.192416:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074749184 00000100:00100000:3.0:1789762948.192423:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074749184 00000100:00100000:0.0:1789762948.192469:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 431 00000100:00100000:0.0:1789762948.192477:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6420:x1876702074749184:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1789762948.192484:0:7227:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762948.192494:0:7227:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1200/0xdf5a7ac59f177938 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f177931 expref: -99 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.192507:0:7227:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1200/0xdf5a7ac59f177938 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f177931 expref: 8 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.192515:0:7227:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1200/0xdf5a7ac59f177938 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177931 expref: 8 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.192520:0:7227:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37e37a7000) 00010000:00010000:0.0:1789762948.192522:0:7227:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762948.192524:0:7227:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d14f1200, rc 0) 00000100:00100000:0.0:1789762948.192551:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074749184:12345-0@lo:101:mdt00_002.0 Request processed in 75us (170us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762948.192556:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 431 00000100:00100000:3.0:1789762948.192575:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11da680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074749184:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1789762948.192586:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d14f1400/0xdf5a7ac59f177931 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177938 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.192594:0:6420:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d14f1400/0xdf5a7ac59f177931 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177938 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762948.192608:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1789762948.192633:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb3480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074749312:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789762948.192655:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762948.192676:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c96ab6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762948.192696:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074749312 00000100:00100000:1.0:1789762948.192701:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074749312 00000100:00100000:1.0:1789762948.192726:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:1.0:1789762948.192734:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074749312:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762948.192810:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074749312:12345-0@lo:1000:mdt00_002.0 Request processed in 77us (152us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762948.192818:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00100000:3.0:1789762948.192825:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb3480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074749312:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789762948.192861:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb3480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074749440:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789762948.192879:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762948.192897:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c96ab6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762948.192911:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074749440 00000100:00100000:1.0:1789762948.192932:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074749440 00000100:00100000:1.0:1789762948.192964:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:1.0:1789762948.192972:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074749440:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762948.193214:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074749440:12345-0@lo:1000:mdt00_002.0 Request processed in 241us (332us total) trans 0 rc 1/1 00000100:00100000:1.0:1789762948.193222:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00100000:3.0:1789762948.193238:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb3480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074749440:0@lo:1000:mdt00_002.0 00010000:00010000:3.0:1789762948.193270:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.193277:0:6420:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.193282:0:6420:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762948.193290:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb3480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074749568:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1789762948.193308:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762948.193323:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c96ab9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789762948.193334:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074749568 00000100:00100000:0.0:1789762948.193338:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074749568 00000100:00100000:0.0:1789762948.193361:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:0.0:1789762948.193366:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074749568:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1789762948.193372:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762948.193380:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1600/0xdf5a7ac59f177946 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f17793f expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.193397:0:6424:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1600/0xdf5a7ac59f177946 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f17793f expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.193404:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1600/0xdf5a7ac59f177946 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f17793f expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.193410:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37e37a7700) 00010000:00010000:0.0:1789762948.193412:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762948.193414:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d14f1600, rc 0) 00000100:00100000:0.0:1789762948.193439:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6420:x1876702074749568:12345-0@lo:101:mdt00_002.0 Request processed in 74us (131us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762948.193444:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:3.0:1789762948.193463:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb3480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074749568:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1789762948.193472:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177946 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.193478:0:6420:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177946 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.193494:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17794d lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.193500:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17794d lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.193508:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17794d lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789762948.194307:0:6420:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].14, 424 off42624 00000040:00080000:3.0:1789762948.194344:0:6420:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].14, 504 off44248 00000004:00080000:3.0:1789762948.194350:0:6420:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37c96ab6c0 with version 30 00000004:00080000:0.0:1789762948.194389:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 30 rpc_version 30 00000020:00100000:0.0:1789762948.194412:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762948.194423:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c36c11dbb80 x1876702074749824/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1744/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762948.194441:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11dbb80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074749824:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762948.194473:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762948.194496:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074749824 00000100:00100000:0.0:1789762948.194498:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d2724000 going to sleep for 11 seconds 00000100:00100000:3.0:1789762948.194501:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074749824 00000100:00100000:3.0:1789762948.194535:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 435 00000100:00100000:3.0:1789762948.194543:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7224:x1876702074749824:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1789762948.195134:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7224:x1876702074749824:12345-0@lo:1000:osp_up0-1.0 Request processed in 592us (660us total) trans 4294967447 rc 0/0 00000100:00100000:3.0:1789762948.195140:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 435 00000100:00100000:0.0:1789762948.195185:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11dbb80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074749824:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1789762948.195194:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17:0x0]: returning size 16384 00010000:00010000:3.0:1789762948.195203:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17794d lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195210:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17794d lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195215:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17794d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195221:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762948.195224:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17794d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195228:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17794d lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195235:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37c72ba500) 00010000:00010000:3.0:1789762948.195237:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762948.195239:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17794d lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195242:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17794d lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195252:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177946 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195258:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f177946 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195263:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f177946 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195275:0:6420:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1789762948.195280:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1789762948.195282:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195291:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.195295:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762948.195297:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.195303:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f177946 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195309:0:6420:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1789762948.195314:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1789762948.195322:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafa00 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:3.0:1789762948.195322:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d14f1400/0xdf5a7ac59f177931 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177938 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.195329:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762948.195333:0:6407:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f177946 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762948.195334:0:6420:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d14f1400/0xdf5a7ac59f177931 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177938 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.195336:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.195363:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caec00 already unused, calling callback (ffffffffc1f70ee0) 00000004:00010000:0.0:1789762948.195365:0:6407:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f177946 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.195368:0:8069:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762948.195386:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:15723:x1876702067552256:12345-192.168.203.37@tcp:36:rm.0 Request processed in 4810us (4895us total) trans 4294967465 rc 0/0 00000100:00100000:3.0:1789762948.195394:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 359 00000004:00010000:1.0:1789762948.195398:0:8069:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.195404:0:6407:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f177946 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.195415:0:6407:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789762948.195431:0:8069:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.195442:0:8069:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1789762948.195446:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 1/0,0 mode: --/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f177946 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762948.195450:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11d9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074749952:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789762948.195456:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.195469:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cafa00/0xdf5a7ac59f17793f lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f177946 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.195479:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caec00/0xdf5a7ac59f177923 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f17792a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762948.195482:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074750080:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762948.195491:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762948.195495:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762948.195501:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762948.195505:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762948.195509:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762948.195512:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762948.195530:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762948.195572:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074749952 00000100:00100000:0.0:1789762948.195578:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074749952 00000100:00100000:2.0:1789762948.195585:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762948.195626:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074750080 00000100:00100000:3.0:1789762948.195630:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074750080 00000100:00100000:0.0:1789762948.195631:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 74 00000100:00100000:0.0:1789762948.195647:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3426:x1876702074749952:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762948.195657:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762948.195670:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1600/0xdf5a7ac59f177946 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f17793f expref: 11 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.195684:0:9878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x0 (ffff9c37e37a7700) 00010000:00010000:0.0:1789762948.195693:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762948.195714:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 75 00000100:00100000:3.0:1789762948.195723:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c90450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3427:x1876702074750080:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789762948.195732:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1789762948.195741:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+10:3426:x1876702074749952:12345-0@lo:103:ldlm_bl.0 Request processed in 98us (258us total) trans 0 rc 0/0 00010000:00010000:3.0:1789762948.195759:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f17792a lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f177923 expref: 10 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762948.195767:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 74 00000100:00100000:2.0:1789762948.195776:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11d9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074749952:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789762948.195795:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762948.195800:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762948.195833:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c90450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3427:x1876702074750080:12345-0@lo:103:ldlm_bl.0 Request processed in 111us (249us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762948.195840:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 75 00000100:00100000:2.0:1789762948.195846:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074750080:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1789762948.608071:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074750208:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762948.608159:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762948.608260:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074750208 00000100:00100000:1.0:1789762948.608267:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074750208 00000100:00100000:1.0:1789762948.608323:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:1.0:1789762948.608335:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074750208:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762948.608379:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762948.608447:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074750208:12345-0@lo:41:osp-pre-1-0.0 Request processed in 116us (290us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762948.608456:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00100000:1.0:1789762948.608516:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074750208:0@lo:41:osp-pre-1-0.0 00000001:00080000:3.0:1789762948.622480:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967447 is committed 00010000:00010000:1.0:1789762948.736041:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1789762949.113561:0:17997:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 422360639121ns, next at 387596343099ns, rc = 0 00000004:00080000:0.0:1789762949.113718:0:17997:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 422360798734ns, next at 422435407932ns, rc = 0 00000004:00080000:3.0:1789762949.116807:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 15 committed 00000004:00080000:1.0:1789762949.128268:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 22 committed 00000001:00080000:1.0:1789762949.128309:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967464 is committed 00000004:00080000:1.0:1789762949.128335:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 21 committed 00010000:00010000:1.0:1789762949.128364:0:7221:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d14f1400/0xdf5a7ac59f177931 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f177938 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.128388:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f17790e lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.128428:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f17790e lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762949.128429:0:7221:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d14f1400/0xdf5a7ac59f177931 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f177938 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.128463:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f17790e lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.128480:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762949.128490:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f17790e lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762949.128494:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:2.0:1789762949.128513:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f17790e lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.128516:0:7221:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d14f1400/0xdf5a7ac59f177931 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f177938 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762949.128529:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11d8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074750336:0@lo:6:osp-syn-0-1.0 00010000:00010000:1.0:1789762949.128552:0:7221:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762949.128562:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17:0x0].0x4009f790 (ffff9c37d3bee100) 00010000:00010000:2.0:1789762949.128571:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762949.128581:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f17790e lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.128597:0:7221:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d14f1400/0xdf5a7ac59f177931 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f177938 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.128598:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f17790e lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x17:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762949.128622:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074750464:0@lo:103:tx_commit_cb.0 00000100:00100000:0.0:1789762949.128639:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:1.0:1789762949.128659:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967465 is committed 00000100:00100000:3.0:1789762949.128715:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762949.128805:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074750464 00000100:00100000:0.0:1789762949.128815:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074750336 00000100:00100000:1.0:1789762949.128816:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074750464 00000100:00100000:0.0:1789762949.128827:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074750336 00000100:00080000:2.0:1789762949.128971:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762949.128974:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 76 00000100:00100000:0.0:1789762949.128990:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:1.0:1789762949.128999:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3426:x1876702074750464:12345-0@lo:103:tx_commit_cb.0 00000100:00100000:0.0:1789762949.129011:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074750336:12345-0@lo:6:osp-syn-0-1.0 00010000:00010000:1.0:1789762949.129020:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762949.129048:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1200/0xdf5a7ac59f177938 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f177931 expref: 7 pid: 7227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1789762949.129049:0:8053:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:14 count 1 00010000:00010000:1.0:1789762949.129090:0:6405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37e37a7000) 00010000:00010000:1.0:1789762949.129101:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00002000:00010000:0.0:1789762949.129115:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 104857600, mtime: 0x6aad9d7a, atime: 0x0, ctime: 0x6aad9d7a, blocks: 0x3200b 00010000:00010000:0.0:1789762949.129126:0:8053:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f177954 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762949.129146:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=422s v=(0 0) 00010000:00010000:0.0:1789762949.129153:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762949.129161:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f177954 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762949.129162:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074750464:12345-0@lo:103:tx_commit_cb.0 Request processed in 172us (449us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762949.129189:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 76 00010000:00010000:0.0:1789762949.129197:0:8053:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f177954 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762949.129204:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074750464:0@lo:103:tx_commit_cb.0 00010000:00010000:0.0:1789762949.129216:0:8053:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f177954 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762949.129232:0:8053:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f177954 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762949.129258:0:8053:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762949.129294:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762949.129308:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f177954 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.129325:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f0a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762949.129332:0:7217:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f177954 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.129347:0:7217:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f177954 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.129375:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f177954 lrc: 1/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.129389:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002a66931/0xdf5a7ac59f177954 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.129431:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762949.129435:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762949.129438:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762949.130038:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074750336:12345-0@lo:6:osp-syn-0-1.0 Request processed in 1034us (1403us total) trans 4294968132 rc 0/0 00000100:00100000:0.0:1789762949.130050:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000004:00080000:1.0:1789762949.130079:0:3424:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c36c11d8a80/2, rc 0, transno 4294968132 00000100:00100000:1.0:1789762949.130093:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11d8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074750336:0@lo:6:osp-syn-0-1.0 00000004:00080000:0.0:1789762949.130243:0:17997:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 422377323211ns, next at 0ns, rc = 0 00000004:00080000:0.0:1789762949.130385:0:17997:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 422377465843ns, next at 369607154287ns, rc = 0 00000004:00080000:1.0:1789762949.131978:0:7221:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 21 committed 00000004:00080000:3.0:1789762949.132304:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 8 committed 00000100:00100000:2.0:1789762949.633149:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074750592:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762949.633260:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762949.633337:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074750592 00000100:00100000:3.0:1789762949.633355:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074750592 00000100:00100000:3.0:1789762949.633443:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:3.0:1789762949.633463:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074750592:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789762949.633529:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:3.0:1789762949.633614:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074750592:12345-0@lo:41:osp-pre-0-1.0 Request processed in 156us (354us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762949.633632:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00100000:2.0:1789762949.633660:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967447 gen 2 00000004:00080000:2.0:1789762949.633670:0:3427:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967447 committed transno 4294967447 00000020:00080000:2.0:1789762949.633680:0:3427:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9c37c96ab840 refcount 2 committed 0 result 0 batchid 14 00000020:00080000:2.0:1789762949.633686:0:3427:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37f7b34100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9c37c96ab6c0 00000020:00080000:2.0:1789762949.633690:0:3427:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x2:0x0].14 00000020:00080000:2.0:1789762949.633695:0:3427:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37f7b34800 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9c37ff83b800 00000020:00080000:2.0:1789762949.633699:0:3427:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x2:0x0].14 00000020:00080000:3.0:1789762949.633765:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 14 committed batchid 13 00000100:00100000:2.0:1789762949.633778:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074750592:0@lo:41:osp-pre-0-1.0 00000020:00080000:3.0:1789762949.634168:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 13 00010000:00010000:0.0:1789762949.760235:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762950.600273:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762950.600328:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762950.600351:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067552512 00000100:00100000:3.0:1789762950.600369:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067552512 00000100:00100000:2.0:1789762950.600409:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762950.600461:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067552640 00000100:00100000:0.0:1789762950.600472:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067552640 00000100:00100000:2.0:1789762950.600482:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762950.600490:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 391 00000100:00100000:1.0:1789762950.600512:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067552768 00000100:00100000:3.0:1789762950.600512:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067552512:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762950.600524:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067552768 00000100:00100000:2.0:1789762950.600558:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762950.600568:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 648 00000100:00100000:0.0:1789762950.600591:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067552640:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762950.600617:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067553024 00000100:00100000:1.0:1789762950.600627:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067553024 00000100:00100000:2.0:1789762950.600636:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762950.600656:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067552512:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 148us (394us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762950.600659:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067552640:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 75us (330us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762950.600670:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067553152 00000100:00100000:1.0:1789762950.600678:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067553152 00000100:00100000:3.0:1789762950.600685:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 391 00000100:00100000:0.0:1789762950.600687:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 648 00000100:00100000:1.0:1789762950.600766:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 649 00000100:00100000:0.0:1789762950.600780:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 650 00000100:00100000:1.0:1789762950.600784:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067552768:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762950.600804:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067553024:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762950.600830:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067552896 00000100:00100000:3.0:1789762950.600843:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067552896 00000100:00100000:1.0:1789762950.600877:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067552768:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 96us (468us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762950.600891:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 649 00000100:00100000:0.0:1789762950.600908:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067553024:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 105us (349us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762950.600927:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 650 00000100:00100000:1.0:1789762950.600940:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 651 00000100:00100000:3.0:1789762950.601005:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 264 00000100:00100000:1.0:1789762950.601022:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067553152:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762950.601026:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067552896:12345-192.168.203.37@tcp:400:kworker.0 00000100:00080000:0.0:1789762950.601084:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762950.601094:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789762950.601103:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762950.601105:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067553152:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 146us (471us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762950.601119:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 651 00000100:00100000:3.0:1789762950.601301:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067552896:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 284us (824us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762950.601310:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 264 00010000:00010000:1.0:1789762950.784111:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762951.680094:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074750976:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762951.680107:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074750720:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762951.680190:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762951.680199:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074751232:0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.680253:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074750720 00000100:00100000:0.0:1789762951.680262:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074750720 00000100:00100000:3.0:1789762951.680264:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074750848:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762951.680281:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762951.680298:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762951.680307:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1adf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074751360:0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.680360:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074750848 00000100:00100000:0.0:1789762951.680365:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074750848 00000100:00100000:2.0:1789762951.680374:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762951.680389:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074751744:0@lo:400:kworker.0 00000100:00100000:3.0:1789762951.680390:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074751488:0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.680409:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 266 00000100:00100000:2.0:1789762951.680416:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074751872:0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.680427:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074750720:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762951.680471:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3754950656 avail 3752853504 00000100:00100000:1.0:1789762951.680491:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762951.680499:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762951.680508:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 267 00000100:00100000:3.0:1789762951.680518:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074750848:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789762951.680550:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762951.680567:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074752256:0@lo:400:kworker.0 00000100:00100000:2.0:1789762951.680573:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074752384:0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.680580:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074750720:12345-0@lo:13:osp-pre-0-0.0 Request processed in 156us (390us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762951.680590:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 266 00000100:00100000:3.0:1789762951.680626:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074750848:12345-0@lo:13:osp-pre-1-1.0 Request processed in 98us (317us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762951.680633:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762951.680641:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789762951.680643:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762951.680651:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762951.680651:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 267 00000100:00100000:0.0:1789762951.680698:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074751744 00000100:00100000:0.0:1789762951.680703:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074751744 00000100:00100000:3.0:1789762951.680729:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074752384 00000100:00100000:3.0:1789762951.680736:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074752384 00000100:00100000:1.0:1789762951.680758:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762951.680761:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074751360 00000100:00100000:2.0:1789762951.680768:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074751360 00000100:00100000:0.0:1789762951.680776:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074751488 00000100:00100000:0.0:1789762951.680782:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074751488 00000100:00100000:3.0:1789762951.680799:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 265 00000100:00100000:1.0:1789762951.680808:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074752000:0@lo:400:kworker.0 00000100:00100000:3.0:1789762951.680817:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074752384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762951.680824:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074751872 00000100:00100000:2.0:1789762951.680829:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074751872 00000100:00100000:0.0:1789762951.680838:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 438 00000100:00100000:0.0:1789762951.680863:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074751744:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762951.680868:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762951.680872:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074752256 00000100:00100000:2.0:1789762951.680876:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074752256 00000100:00100000:3.0:1789762951.680893:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074752384:12345-0@lo:400:kworker.0 Request processed in 81us (265us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762951.680905:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074752512:0@lo:400:kworker.0 00000100:00100000:3.0:1789762951.680906:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 265 00000100:00100000:2.0:1789762951.680907:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074752000 00000100:00100000:2.0:1789762951.680962:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074752000 00000100:00100000:0.0:1789762951.680986:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074751744:12345-0@lo:400:kworker.0 Request processed in 136us (493us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762951.681007:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 438 00000100:00100000:2.0:1789762951.681022:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 652 00000100:00100000:1.0:1789762951.681023:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762951.681025:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074752384:0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.681034:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:2.0:1789762951.681050:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff615c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074751360:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.681051:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074751488:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762951.681103:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074750720:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762951.681118:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 653 00000100:00100000:0.0:1789762951.681134:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074751488:12345-0@lo:400:kworker.0 Request processed in 85us (378us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762951.681136:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ff614a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074751872:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762951.681147:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074750848:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762951.681155:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00100000:2.0:1789762951.681156:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff615c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074751360:12345-0@lo:400:kworker.0 Request processed in 106us (776us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762951.681182:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 652 00000100:00100000:3.0:1789762951.681203:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ff614a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074751872:12345-0@lo:400:kworker.0 Request processed in 73us (705us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762951.681206:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074751232 00000100:00100000:0.0:1789762951.681212:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074751232 00000100:00100000:3.0:1789762951.681217:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 653 00000100:00100000:2.0:1789762951.681227:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 654 00000100:00100000:1.0:1789762951.681232:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074751488:0@lo:400:kworker.0 00000100:00100000:2.0:1789762951.681248:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074752256:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762951.681253:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 655 00000100:00100000:1.0:1789762951.681255:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:3.0:1789762951.681267:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074752000:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762951.681271:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074751104:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762951.681272:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:2.0:1789762951.681295:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074752256:12345-0@lo:400:kworker.0 Request processed in 50us (675us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762951.681296:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074751232:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762951.681310:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 654 00000100:00100000:1.0:1789762951.681336:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762951.681357:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074752000:12345-0@lo:400:kworker.0 Request processed in 90us (495us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762951.681370:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 655 00000100:00100000:0.0:1789762951.681384:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3426:x1876702074751232:12345-0@lo:400:kworker.0 Request processed in 88us (1099us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762951.681384:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1af100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074751616:0@lo:400:kworker.0 00000100:00100000:2.0:1789762951.681397:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1adf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074751360:0@lo:400:kworker.0 00000100:00100000:3.0:1789762951.681401:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074751104 00000100:00100000:0.0:1789762951.681412:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00100000:3.0:1789762951.681412:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074751104 00000100:00100000:2.0:1789762951.681434:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074751872:0@lo:400:kworker.0 00000100:00100000:1.0:1789762951.681444:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762951.681467:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 268 00000100:00100000:3.0:1789762951.681482:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074751104:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762951.681486:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074752128:0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.681492:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074752512 00000100:00100000:2.0:1789762951.681494:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074751616 00000100:00100000:0.0:1789762951.681497:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074752512 00002000:00000024:3.0:1789762951.681501:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762951.681502:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074751616 00000100:00100000:1.0:1789762951.681527:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762951.681552:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 266 00000100:00100000:0.0:1789762951.681568:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074752512:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762951.681568:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074752128 00000100:00100000:2.0:1789762951.681576:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074752128 00000100:00100000:1.0:1789762951.681577:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37ff617480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074752640:0@lo:400:kworker.0 00000100:00100000:3.0:1789762951.681579:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074751104:12345-0@lo:13:osp-pre-0-1.0 Request processed in 95us (237us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762951.681605:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 268 00000100:00100000:2.0:1789762951.681622:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 656 00000100:00100000:0.0:1789762951.681626:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074752512:12345-0@lo:400:kworker.0 Request processed in 61us (627us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762951.681635:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 266 00000100:00100000:2.0:1789762951.681651:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074751616:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762951.681655:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762951.681657:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789762951.681665:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762951.681668:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789762951.681671:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762951.681737:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074751744:0@lo:400:kworker.0 00000100:00100000:1.0:1789762951.681747:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074752000:0@lo:400:kworker.0 00000100:00100000:2.0:1789762951.681763:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074751616:12345-0@lo:400:kworker.0 Request processed in 125us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762951.681767:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074750976 00000100:00100000:0.0:1789762951.681771:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074752256:0@lo:400:kworker.0 00000100:00100000:3.0:1789762951.681773:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074750976 00000100:00100000:2.0:1789762951.681779:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 656 00000100:00100000:1.0:1789762951.681782:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074752512:0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.681816:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:1065be24-cabd-445e-9b93-f5987588951d:3426:1876702074751232:0@lo:400:kworker.0 00000100:00100000:2.0:1789762951.681822:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 657 00000100:00100000:3.0:1789762951.681831:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 269 00000100:00100000:2.0:1789762951.681838:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074752128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762951.681851:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074750976:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762951.681853:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074751104:0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1789762951.681877:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00080000:0.0:1789762951.681880:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762951.681896:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1af100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074751616:0@lo:400:kworker.0 00000100:00100000:2.0:1789762951.681899:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074752128:12345-0@lo:400:kworker.0 Request processed in 66us (376us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762951.681956:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 657 00000100:00100000:1.0:1789762951.681958:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074752128:0@lo:400:kworker.0 00000100:00100000:3.0:1789762951.681969:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074750976:12345-0@lo:13:osp-pre-1-0.0 Request processed in 127us (320us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762951.681980:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 269 00000100:00100000:0.0:1789762951.681992:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074752640 00000100:00100000:0.0:1789762951.681999:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074752640 00000100:00100000:0.0:1789762951.682058:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 267 00000100:00100000:3.0:1789762951.682063:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074750976:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762951.682070:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074752640:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.682110:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074752640:12345-0@lo:400:kworker.0 Request processed in 47us (473us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762951.682115:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 267 00000100:00100000:1.0:1789762951.682153:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37ff617480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074752640:0@lo:400:kworker.0 00010000:00010000:1.0:1789762951.808186:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762952.832254:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762953.728109:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789762953.728136:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074752768:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762953.728220:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762953.728281:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074752768 00000100:00100000:2.0:1789762953.728295:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074752768 00000100:00100000:2.0:1789762953.728371:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 440 00000100:00100000:2.0:1789762953.728387:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074752768:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789762953.728436:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762953.728562:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074752768:12345-0@lo:41:osp-pre-1-0.0 Request processed in 179us (339us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762953.728572:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 440 00000100:00100000:2.0:1789762953.728650:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074752768:0@lo:41:osp-pre-1-0.0 00000001:00080000:2.0:1789762953.747018:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968132 is committed 00010000:00010000:0.0:1789762953.856102:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:1.0:1789762954.248085:0:7221:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 14 batchid updated 00000020:00080000:3.0:1789762954.248112:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 14 00000004:00080000:3.0:1789762954.248172:0:7233:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37d875f3c0 with version 31 00000020:00080000:3.0:1789762954.248185:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 14 cancel update log [0x200000401:0x2:0x0].14: rc = 0 00000004:00080000:0.0:1789762954.248207:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 31 rpc_version 31 00000020:00100000:0.0:1789762954.248250:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762954.248261:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c36c11d9880 x1876702074753024/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762954.248285:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11d9880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074753024:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762954.248373:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762954.248422:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074753024 00000100:00100000:1.0:1789762954.248435:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074753024 00000100:00100000:0.0:1789762954.248464:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37c55c8f00 going to sleep for 11 seconds 00000020:00080000:3.0:1789762954.248489:0:7233:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 14 cancel update log [0x240000400:0x2:0x0].14: rc = 0 00000100:00100000:1.0:1789762954.248499:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 441 00000020:00080000:3.0:1789762954.248499:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 14 00000100:00100000:1.0:1789762954.248513:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:7224:x1876702074753024:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1789762954.248741:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7224:x1876702074753024:12345-0@lo:1000:osp_up0-1.0 Request processed in 231us (368us total) trans 4294967448 rc 0/0 00000100:00100000:1.0:1789762954.248748:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 441 00000100:00100000:0.0:1789762954.248795:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11d9880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074753024:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1789762954.752126:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074753152:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762954.752213:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762954.752255:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074753152 00000100:00100000:3.0:1789762954.752264:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074753152 00000100:00100000:3.0:1789762954.752319:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 442 00000100:00100000:3.0:1789762954.752329:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3427:x1876702074753152:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789762954.752371:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:3.0:1789762954.752425:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3427:x1876702074753152:12345-0@lo:41:osp-pre-0-1.0 Request processed in 98us (211us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762954.752433:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 442 00000100:00100000:2.0:1789762954.752472:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074753152:0@lo:41:osp-pre-0-1.0 00010000:00010000:0.0:1789762954.880110:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762955.720103:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762955.720147:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762955.720180:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762955.720204:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067553280 00000100:00100000:2.0:1789762955.720205:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762955.720219:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067553280 00000100:00100000:2.0:1789762955.720246:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762955.720247:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067553664 00000100:00100000:3.0:1789762955.720259:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067553664 00000100:00100000:1.0:1789762955.720274:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067553408 00000100:00100000:1.0:1789762955.720287:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067553408 00000100:00100000:2.0:1789762955.720288:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762955.720322:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 393 00000100:00100000:2.0:1789762955.720336:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067553536 00000100:00100000:0.0:1789762955.720345:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067553280:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762955.720346:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067553536 00000100:00100000:3.0:1789762955.720346:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 268 00000100:00100000:1.0:1789762955.720367:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067553920 00000100:00100000:3.0:1789762955.720369:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067553664:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762955.720379:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067553920 00000100:00100000:2.0:1789762955.720400:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067553792 00000100:00100000:2.0:1789762955.720406:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067553792 00000100:00100000:0.0:1789762955.720447:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067553280:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 104us (357us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762955.720454:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 658 00000100:00100000:3.0:1789762955.720457:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067553664:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 80us (259us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762955.720467:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 393 00000100:00100000:2.0:1789762955.720470:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 659 00000100:00100000:1.0:1789762955.720477:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067553408:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762955.720488:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 268 00000100:00100000:2.0:1789762955.720514:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067553536:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762955.720534:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067553408:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 64us (381us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762955.720539:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 660 00000100:00100000:1.0:1789762955.720553:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 658 00000100:00100000:0.0:1789762955.720562:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067553920:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762955.720571:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067553536:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 80us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762955.720573:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 661 00000100:00100000:2.0:1789762955.720595:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 659 00000100:00100000:3.0:1789762955.720597:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067553792:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762955.720616:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067553920:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 62us (369us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762955.720626:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 660 00000100:00080000:0.0:1789762955.720659:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762955.720666:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067553792:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 71us (371us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762955.720667:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762955.720676:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762955.720682:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 661 00010000:00010000:1.0:1789762955.904273:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762956.800067:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e3bb2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074753536:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762956.800075:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074753280:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762956.800082:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074753408:0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.800161:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762956.800163:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762956.800195:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074753664:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762956.800202:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074753408 00000100:00100000:0.0:1789762956.800209:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074753408 00000100:00100000:1.0:1789762956.800218:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074753792:0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.800226:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074753920:0@lo:400:kworker.0 00000100:00100000:2.0:1789762956.800255:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762956.800256:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:3.0:1789762956.800269:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762956.800270:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074753408:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.800270:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762956.800297:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074754176:0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.800303:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074754432:0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.800315:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074754304:0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.800325:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3425:x1876702074753408:12345-0@lo:400:kworker.0 Request processed in 60us (165us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762956.800331:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:2.0:1789762956.800351:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762956.800352:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762956.800364:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074753792 00000100:00100000:1.0:1789762956.800368:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074753792 00000100:00100000:0.0:1789762956.800378:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762956.800391:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074754944:0@lo:400:kworker.0 00000100:00100000:2.0:1789762956.800400:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074754688:0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.800405:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074754432 00000100:00100000:1.0:1789762956.800409:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074754432 00000100:00100000:0.0:1789762956.800417:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074754048:0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.800425:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762956.800451:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 662 00000100:00100000:2.0:1789762956.800455:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762956.800462:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074753280 00000100:00100000:3.0:1789762956.800464:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1065be24-cabd-445e-9b93-f5987588951d:3425:1876702074753408:0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.800467:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074753792:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.800468:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074753280 00000100:00100000:2.0:1789762956.800497:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074755200:0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.800510:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074753664 00000100:00100000:0.0:1789762956.800514:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074753664 00000100:00100000:1.0:1789762956.800521:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074753792:12345-0@lo:400:kworker.0 Request processed in 57us (256us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762956.800522:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762956.800531:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 662 00000100:00100000:2.0:1789762956.800545:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762956.800546:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074753536 00000100:00100000:0.0:1789762956.800550:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074753536 00000100:00100000:1.0:1789762956.800553:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074754688 00000100:00100000:3.0:1789762956.800555:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074754816:0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.800558:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074754688 00000100:00100000:0.0:1789762956.800576:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 270 00000100:00100000:0.0:1789762956.800588:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074753280:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762956.800588:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074754304 00000100:00100000:1.0:1789762956.800594:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074754304 00000100:00100000:3.0:1789762956.800595:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762956.800612:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 271 00000100:00100000:1.0:1789762956.800623:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 663 00000100:00100000:3.0:1789762956.800623:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074755328:0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789762956.800625:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762956.800631:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074753664:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762956.800639:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074754432:12345-0@lo:400:kworker.0 00002000:00000024:2.0:1789762956.800666:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762956.800677:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762956.800696:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074753280:12345-0@lo:13:osp-pre-1-0.0 Request processed in 96us (524us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762956.800704:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 270 00000100:00100000:0.0:1789762956.800720:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074755328 00000100:00100000:1.0:1789762956.800721:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074754432:12345-0@lo:400:kworker.0 Request processed in 84us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762956.800725:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074755328 00000100:00100000:1.0:1789762956.800731:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 663 00000100:00100000:3.0:1789762956.800734:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074753280:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762956.800755:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074753664:12345-0@lo:13:osp-pre-0-1.0 Request processed in 131us (501us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762956.800757:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074753792:0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.800760:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 272 00000100:00100000:1.0:1789762956.800761:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 664 00000100:00100000:2.0:1789762956.800776:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 271 00000100:00100000:0.0:1789762956.800783:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074753536:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762956.800784:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074754688:12345-0@lo:400:kworker.0 00002000:00000024:0.0:1789762956.800805:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762956.800806:0:8055:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 273 00000100:00100000:3.0:1789762956.800809:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074754432:0@lo:400:kworker.0 00000100:00100000:2.0:1789762956.800824:0:8055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074755328:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762956.800834:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074754688:12345-0@lo:400:kworker.0 Request processed in 60us (386us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762956.800840:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 664 00002000:00000024:2.0:1789762956.800842:0:8055:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762956.800848:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074753536:12345-0@lo:13:osp-pre-0-0.0 Request processed in 71us (467us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762956.800854:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 665 00000100:00100000:0.0:1789762956.800858:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 272 00000100:00100000:3.0:1789762956.800867:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074754304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762956.800903:0:8055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074755328:12345-0@lo:13:osp-pre-1-1.0 Request processed in 84us (224us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762956.800904:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762956.800905:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074754944 00000100:00100000:0.0:1789762956.800911:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074754944 00000100:00080000:1.0:1789762956.800912:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762956.800913:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074754304:12345-0@lo:400:kworker.0 Request processed in 48us (393us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762956.800941:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789762956.800946:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762956.800947:0:8055:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 273 00000100:00100000:3.0:1789762956.800950:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 665 00000100:00100000:1.0:1789762956.800965:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762956.800977:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074754048 00000100:00100000:3.0:1789762956.800981:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074754048 00000100:00100000:0.0:1789762956.800983:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074755200 00000100:00100000:0.0:1789762956.800988:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074755200 00000100:00100000:1.0:1789762956.800995:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074754560:0@lo:400:kworker.0 00000100:00100000:2.0:1789762956.801003:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294968132 gen 2 00000004:00080000:2.0:1789762956.801011:0:3427:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c36c11d8a80, transno 4294968132 00000100:00100000:0.0:1789762956.801020:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074754816 00000100:00100000:3.0:1789762956.801020:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 666 00000100:00100000:0.0:1789762956.801026:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074754816 00000100:00100000:1.0:1789762956.801026:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762956.801036:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074754048:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762956.801039:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074753664:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762956.801055:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 269 00000100:00100000:1.0:1789762956.801055:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9629f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074755072:0@lo:400:kworker.0 00000100:00100000:2.0:1789762956.801066:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074754688:0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.801071:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074754944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.801080:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074754048:12345-0@lo:400:kworker.0 Request processed in 49us (118us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762956.801092:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 666 00000100:00100000:1.0:1789762956.801096:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762956.801112:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074754560 00000100:00100000:3.0:1789762956.801116:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074754560 00000100:00100000:0.0:1789762956.801125:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074754944:12345-0@lo:400:kworker.0 Request processed in 58us (699us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762956.801131:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 269 00000100:00100000:1.0:1789762956.801141:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e3bb2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074753536:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762956.801151:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074755072 00000100:00100000:3.0:1789762956.801155:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 667 00000100:00100000:0.0:1789762956.801157:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074755072 00000100:00100000:3.0:1789762956.801167:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074754560:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.801174:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074754048:0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.801184:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 270 00000100:00100000:0.0:1789762956.801196:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074755200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.801216:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074754560:12345-0@lo:400:kworker.0 Request processed in 53us (193us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762956.801223:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 667 00000100:00100000:1.0:1789762956.801229:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 271 00000100:00100000:1.0:1789762956.801240:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074754816:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.801243:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074755200:12345-0@lo:400:kworker.0 Request processed in 49us (701us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762956.801245:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074753920 00000100:00100000:3.0:1789762956.801251:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074753920 00000100:00100000:0.0:1789762956.801255:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 270 00000100:00100000:0.0:1789762956.801274:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 272 00000100:00100000:0.0:1789762956.801283:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074755072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.801286:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074754176 00000100:00100000:3.0:1789762956.801290:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074754176 00000100:00100000:1.0:1789762956.801299:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074754816:12345-0@lo:400:kworker.0 Request processed in 59us (706us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762956.801306:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 271 00000100:00100000:3.0:1789762956.801321:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:0.0:1789762956.801328:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074755072:12345-0@lo:400:kworker.0 Request processed in 46us (237us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762956.801333:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e3bb0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3425:x1876702074753920:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.801339:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 272 00000100:00100000:1.0:1789762956.801357:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074754304:0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.801375:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074754816:0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.801380:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e3bb0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3425:x1876702074753920:12345-0@lo:400:kworker.0 Request processed in 51us (1116us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762956.801387:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:1.0:1789762956.801394:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074755328:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762956.801400:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:0.0:1789762956.801411:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9628000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074754176:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789762956.801437:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762956.801437:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074754560:0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.801456:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9628000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074754176:12345-0@lo:400:kworker.0 Request processed in 47us (1109us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762956.801461:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9629f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074755072:0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.801466:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:1.0:1789762956.801470:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074753920:0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.801490:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074754944:0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.801522:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074754176:0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.801534:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074755200:0@lo:400:kworker.0 00010000:00010000:2.0:1789762956.928085:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762957.952145:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762958.848079:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9629180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074755456:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762958.848160:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762958.848220:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074755456 00000100:00100000:3.0:1789762958.848229:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074755456 00000100:00100000:3.0:1789762958.848288:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 445 00000100:00100000:3.0:1789762958.848300:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9628a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074755456:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789762958.848340:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:3.0:1789762958.848406:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9628a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074755456:12345-0@lo:41:osp-pre-1-0.0 Request processed in 109us (246us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762958.848412:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 445 00000100:00100000:3.0:1789762958.848467:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9629180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074755456:0@lo:41:osp-pre-1-0.0 00010000:00010000:0.0:1789762958.976104:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789762959.369693:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967448 is committed 00000100:00100000:3.0:1789762959.872064:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789762959.872090:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074755584:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789762959.872152:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762959.872191:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074755584 00000100:00100000:0.0:1789762959.872200:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074755584 00000100:00100000:0.0:1789762959.872252:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 446 00000100:00100000:0.0:1789762959.872261:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074755584:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762959.872293:0:6423:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:0.0:1789762959.872338:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074755584:12345-0@lo:41:osp-pre-0-1.0 Request processed in 78us (186us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762959.872343:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 446 00000100:00100000:3.0:1789762959.872379:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967448 gen 2 00000004:00080000:3.0:1789762959.872386:0:3427:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967448 committed transno 4294967448 00000100:00100000:3.0:1789762959.872408:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074755584:0@lo:41:osp-pre-0-1.0 00010000:00010000:1.0:1789762960.000100:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762960.840033:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762960.840070:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762960.840081:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067554048 00000100:00100000:2.0:1789762960.840091:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762960.840096:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067554048 00000100:00100000:1.0:1789762960.840128:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067554176 00000100:00100000:1.0:1789762960.840136:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067554176 00000100:00100000:2.0:1789762960.840136:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762960.840155:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 395 00000100:00100000:0.0:1789762960.840158:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067554304 00000100:00100000:0.0:1789762960.840165:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067554304 00000100:00100000:3.0:1789762960.840167:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067554048:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762960.840174:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762960.840202:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067554560 00000100:00100000:1.0:1789762960.840207:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067554560 00000100:00100000:0.0:1789762960.840219:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 668 00000100:00100000:0.0:1789762960.840234:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9629c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067554176:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762960.840242:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:0607efc5-93ac-4424-8fa4-2e162e139892+13:2398:x1876702067554048:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 74us (211us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762960.840243:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 669 00000100:00100000:2.0:1789762960.840252:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762960.840259:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 395 00000100:00100000:1.0:1789762960.840261:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067554304:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762960.840287:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9629c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067554176:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 56us (214us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762960.840294:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 668 00000100:00100000:1.0:1789762960.840299:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067554304:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 44us (206us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762960.840306:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 669 00000100:00100000:0.0:1789762960.840312:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067554688 00000100:00100000:0.0:1789762960.840317:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067554688 00000100:00100000:1.0:1789762960.840329:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 670 00000100:00100000:1.0:1789762960.840343:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9628700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067554560:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762960.840349:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 671 00000100:00100000:0.0:1789762960.840360:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067554688:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762960.840363:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067554432 00000100:00100000:3.0:1789762960.840369:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067554432 00000100:00100000:1.0:1789762960.840380:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9628700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2398:x1876702067554560:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 40us (209us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762960.840390:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 670 00000100:00100000:0.0:1789762960.840395:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2397:x1876702067554688:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 37us (148us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762960.840403:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 671 00000100:00080000:1.0:1789762960.840418:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789762960.840423:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789762960.840426:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762960.840439:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 273 00000100:00100000:2.0:1789762960.840449:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37ffcc8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067554432:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762960.840488:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37ffcc8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067554432:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 40us (359us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762960.840493:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 273 00010000:00010000:2.0:1789762961.024092:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762961.920132:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074756096:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762961.920151:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074755968:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762961.920193:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074755840:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762961.920219:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762961.920243:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9629880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074755712:0@lo:400:kworker.0 00000100:00100000:1.0:1789762961.920264:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762961.920275:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762961.920298:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762961.920329:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074756480:0@lo:400:kworker.0 00000100:00100000:1.0:1789762961.920337:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074756224:0@lo:400:kworker.0 00000100:00100000:2.0:1789762961.920347:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9628380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074756352:0@lo:400:kworker.0 00000100:00100000:3.0:1789762961.920366:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9629180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074756608:0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.920380:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762961.920391:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074756096 00000100:00100000:2.0:1789762961.920396:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762961.920399:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074756096 00000100:00100000:0.0:1789762961.920447:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9629f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074756992:0@lo:400:kworker.0 00000100:00100000:2.0:1789762961.920452:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074756864:0@lo:400:kworker.0 00000100:00100000:3.0:1789762961.920453:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762961.920454:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074755840 00000100:00100000:1.0:1789762961.920464:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074755840 00000100:00100000:1.0:1789762961.920498:0:13091:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074755968 00000100:00100000:1.0:1789762961.920502:0:13091:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074755968 00000100:00100000:3.0:1789762961.920514:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074757120:0@lo:400:kworker.0 00000100:00100000:1.0:1789762961.920530:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 274 00000100:00100000:2.0:1789762961.920532:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762961.920551:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 275 00000100:00100000:1.0:1789762961.920551:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074756096:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762961.920571:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074757376:0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.920578:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074755840:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789762961.920586:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762961.920586:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:0.0:1789762961.920621:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762961.920623:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762961.920630:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb591f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074757632:0@lo:400:kworker.0 00000100:00100000:1.0:1789762961.920694:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074756096:12345-0@lo:13:osp-pre-1-0.0 Request processed in 146us (475us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762961.920700:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074756352 00000100:00100000:2.0:1789762961.920711:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074756352 00000100:00100000:1.0:1789762961.920712:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 274 00000100:00100000:3.0:1789762961.920717:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762961.920721:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074755840:12345-0@lo:13:osp-pre-1-1.0 Request processed in 147us (448us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762961.920736:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 275 00000100:00100000:1.0:1789762961.920769:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:2.0:1789762961.920774:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074756608 00000100:00100000:2.0:1789762961.920780:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074756608 00000100:00100000:3.0:1789762961.920783:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789762961.920787:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7ad9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074755968:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762961.920796:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36cb592a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074757760:0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.920809:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074755712 00002000:00000024:1.0:1789762961.920813:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762961.920816:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074755712 00000100:00100000:2.0:1789762961.920818:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 447 00000100:00100000:2.0:1789762961.920830:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb590380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074756352:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762961.920842:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762961.920860:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7ad9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074755968:12345-0@lo:13:osp-pre-0-0.0 Request processed in 82us (589us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762961.920868:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00000100:00100000:0.0:1789762961.920893:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:0.0:1789762961.920914:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074755712:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762961.920962:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb590380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074756352:12345-0@lo:400:kworker.0 Request processed in 89us (523us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762961.920972:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 447 00000100:00100000:3.0:1789762961.920995:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074755968:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762961.921006:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:2.0:1789762961.921016:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d9a34850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074756608:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.921073:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074755712:12345-0@lo:400:kworker.0 Request processed in 168us (775us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762961.921093:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:2.0:1789762961.921094:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d9a34850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074756608:12345-0@lo:400:kworker.0 Request processed in 75us (644us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762961.921102:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:3.0:1789762961.921127:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9629180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074756608:0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.921164:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074757376 00000100:00100000:0.0:1789762961.921171:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074757376 00000100:00100000:0.0:1789762961.921243:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074757632 00000100:00100000:3.0:1789762961.921243:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074756480 00000100:00100000:0.0:1789762961.921255:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074757632 00000100:00100000:3.0:1789762961.921256:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074756480 00000100:00100000:0.0:1789762961.921303:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074757760 00000100:00100000:0.0:1789762961.921311:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074757760 00000100:00100000:3.0:1789762961.921350:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074756864 00000100:00100000:0.0:1789762961.921359:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 274 00000100:00100000:3.0:1789762961.921359:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074756864 00000100:00080000:1.0:1789762961.921374:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762961.921380:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074757376:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789762961.921385:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762961.921394:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762961.921442:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074757120 00000100:00100000:1.0:1789762961.921443:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 275 00000100:00100000:0.0:1789762961.921453:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074757376:12345-0@lo:400:kworker.0 Request processed in 72us (824us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762961.921454:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074757120 00000100:00100000:1.0:1789762961.921462:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074757632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.921475:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 274 00000100:00100000:0.0:1789762961.921518:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:1.0:1789762961.921535:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074757632:12345-0@lo:400:kworker.0 Request processed in 75us (830us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762961.921541:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074757760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762961.921548:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 672 00000100:00100000:1.0:1789762961.921559:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 275 00000100:00100000:3.0:1789762961.921571:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074756480:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762961.921601:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 673 00000100:00100000:1.0:1789762961.921615:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb593b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074756864:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.921624:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074757760:12345-0@lo:400:kworker.0 Request processed in 92us (786us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762961.921636:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00000100:00100000:2.0:1789762961.921666:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9629880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074755712:0@lo:400:kworker.0 00000100:00100000:3.0:1789762961.921670:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3425:x1876702074756480:12345-0@lo:400:kworker.0 Request processed in 105us (1291us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762961.921682:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 672 00000100:00100000:2.0:1789762961.921694:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9628380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074756352:0@lo:400:kworker.0 00000100:00100000:1.0:1789762961.921700:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb593b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074756864:12345-0@lo:400:kworker.0 Request processed in 85us (1177us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762961.921705:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 674 00000100:00100000:1.0:1789762961.921717:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 673 00000100:00100000:2.0:1789762961.921722:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074756864:0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.921728:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074757120:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762961.921743:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074757376:0@lo:400:kworker.0 00000100:00080000:3.0:1789762961.921755:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762961.921762:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762961.921809:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074757504:0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.921815:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074757120:12345-0@lo:400:kworker.0 Request processed in 92us (1234us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762961.921819:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb592a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074757760:0@lo:400:kworker.0 00000100:00100000:2.0:1789762961.921820:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074756992 00000100:00100000:2.0:1789762961.921830:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074756992 00000100:00100000:1.0:1789762961.921846:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762961.921852:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 674 00000100:00100000:3.0:1789762961.921875:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074757120:0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.921905:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 675 00000100:00100000:3.0:1789762961.921912:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36cb591f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074757632:0@lo:400:kworker.0 00000100:00100000:1.0:1789762961.921915:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074756096:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762961.921952:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074756992:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762961.921962:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789762961.922017:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3425:1876702074756480:0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.922037:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074756992:12345-0@lo:400:kworker.0 Request processed in 120us (275us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762961.922039:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9629f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074756992:0@lo:400:kworker.0 00000100:00100000:2.0:1789762961.922042:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762961.922059:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 675 00000100:00100000:2.0:1789762961.922083:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074756736:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762961.922092:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074756224 00000100:00100000:1.0:1789762961.922097:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074757504 00000100:00100000:3.0:1789762961.922100:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074756224 00000100:00100000:1.0:1789762961.922102:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074757504 00000100:00100000:2.0:1789762961.922109:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762961.922136:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074757248:0@lo:400:kworker.0 00000100:00100000:1.0:1789762961.922144:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 277 00000100:00100000:1.0:1789762961.922154:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074757504:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762961.922169:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762961.922181:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074756736 00000100:00100000:3.0:1789762961.922185:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074757248 00000100:00100000:0.0:1789762961.922189:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074756736 00000100:00100000:3.0:1789762961.922192:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074757248 00000100:00100000:1.0:1789762961.922219:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074757504:12345-0@lo:400:kworker.0 Request processed in 66us (380us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762961.922228:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074755840:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762961.922229:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 277 00000100:00100000:0.0:1789762961.922249:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 277 00000100:00100000:0.0:1789762961.922264:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074756736:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762961.922277:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 676 00002000:00000024:0.0:1789762961.922279:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762961.922293:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 677 00000100:00080000:2.0:1789762961.922300:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789762961.922300:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074756224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762961.922314:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074757248:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.922338:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074756736:12345-0@lo:13:osp-pre-0-1.0 Request processed in 75us (224us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762961.922349:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 277 00000100:00100000:2.0:1789762961.922377:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074756736:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762961.922389:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074757248:12345-0@lo:400:kworker.0 Request processed in 80us (224us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762961.922397:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 677 00000100:00100000:3.0:1789762961.922397:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3427:x1876702074756224:12345-0@lo:400:kworker.0 Request processed in 107us (356us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762961.922407:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074757248:0@lo:400:kworker.0 00000100:00100000:0.0:1789762961.922417:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074757504:0@lo:400:kworker.0 00000100:00100000:3.0:1789762961.922431:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 676 00000100:00100000:2.0:1789762961.922457:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3427:1876702074756224:0@lo:400:kworker.0 00010000:00010000:0.0:1789762962.048127:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762963.072119:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762963.968100:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789762963.968122:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074757888:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789762963.968190:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762963.968232:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074757888 00000100:00100000:3.0:1789762963.968240:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074757888 00000100:00100000:3.0:1789762963.968299:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:3.0:1789762963.968312:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074757888:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789762963.968346:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:3.0:1789762963.968412:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074757888:12345-0@lo:41:osp-pre-1-0.0 Request processed in 102us (221us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762963.968421:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:0.0:1789762963.968455:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074757888:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1789762964.096161:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762964.992107:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762964.992126:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074758016:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762964.992188:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762964.992241:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074758016 00000100:00100000:0.0:1789762964.992249:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074758016 00000100:00100000:0.0:1789762964.992292:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:0.0:1789762964.992303:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d69500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074758016:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762964.992336:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:0.0:1789762964.992396:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d69500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074758016:12345-0@lo:41:osp-pre-0-1.0 Request processed in 94us (206us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762964.992403:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00100000:2.0:1789762964.992427:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074758016:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1789762965.120093:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762965.960350:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762965.960384:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762965.960405:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762965.960413:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067554816 00000100:00100000:0.0:1789762965.960425:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067554816 00000100:00100000:3.0:1789762965.960434:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762965.960437:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067554944 00000100:00100000:2.0:1789762965.960444:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067554944 00000100:00100000:3.0:1789762965.960463:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762965.960468:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067555072 00000100:00100000:1.0:1789762965.960475:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067555072 00000100:00100000:0.0:1789762965.960484:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 397 00000100:00100000:0.0:1789762965.960498:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067554816:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762965.960501:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067555328 00000100:00100000:3.0:1789762965.960506:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762965.960508:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067555328 00000100:00100000:1.0:1789762965.960518:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067555456 00000100:00100000:1.0:1789762965.960523:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067555456 00000100:00100000:3.0:1789762965.960550:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 678 00000100:00100000:2.0:1789762965.960558:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 679 00000100:00100000:3.0:1789762965.960564:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067554944:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762965.960573:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067554816:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 76us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762965.960575:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067555072:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762965.960581:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 680 00000100:00100000:0.0:1789762965.960592:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 397 00000100:00100000:1.0:1789762965.960599:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067555328:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762965.960608:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067554944:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 48us (220us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762965.960628:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 678 00000100:00100000:2.0:1789762965.960638:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067555072:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 67us (230us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762965.960648:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 679 00000100:00100000:3.0:1789762965.960656:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 681 00000100:00100000:1.0:1789762965.960657:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067555328:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 59us (191us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762965.960669:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067555456:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762965.960671:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 680 00000100:00080000:2.0:1789762965.960678:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762965.960685:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762965.960706:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067555456:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 41us (202us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762965.960712:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 681 00000100:00100000:1.0:1789762965.960739:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067555200 00000100:00100000:1.0:1789762965.960744:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067555200 00000100:00080000:2.0:1789762965.960772:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762965.960789:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 278 00000100:00100000:1.0:1789762965.960797:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067555200:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762965.960833:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067555200:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 38us (402us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762965.960838:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 278 00010000:00010000:0.0:1789762966.144138:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762967.040084:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074758272:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762967.040089:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074758144:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762967.040104:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37fddd2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074758656:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789762967.040163:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762967.040173:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074758528:0@lo:400:kworker.0 00000100:00100000:1.0:1789762967.040198:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762967.040223:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762967.040236:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074758144 00000100:00100000:0.0:1789762967.040242:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d68700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074758784:0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.040243:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074758144 00000100:00100000:1.0:1789762967.040247:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074759168:0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.040259:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074759040:0@lo:400:kworker.0 00000100:00100000:1.0:1789762967.040302:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762967.040304:0:8056:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074758656 00000100:00100000:0.0:1789762967.040309:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762967.040310:0:8056:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074758656 00000100:00100000:3.0:1789762967.040323:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074758528 00000100:00100000:3.0:1789762967.040329:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074758528 00000100:00100000:1.0:1789762967.040346:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074759680:0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.040346:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 278 00000100:00100000:0.0:1789762967.040350:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074759296:0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.040362:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074758144:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789762967.040367:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:3.0:1789762967.040378:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074758528:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762967.040387:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:2.0:1789762967.040397:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762967.040408:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074758784 00000100:00100000:0.0:1789762967.040413:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074758784 00000100:00100000:1.0:1789762967.040428:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074760192:0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.040436:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074758528:12345-0@lo:400:kworker.0 Request processed in 58us (212us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762967.040443:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:2.0:1789762967.040451:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074758144:12345-0@lo:13:osp-pre-0-1.0 Request processed in 91us (288us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762967.040453:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074759680 00000100:00100000:0.0:1789762967.040458:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074759680 00000100:00100000:2.0:1789762967.040465:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 278 00000100:00100000:1.0:1789762967.040472:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1789762967.040488:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789762967.040495:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762967.040496:0:8056:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 279 00000100:00080000:3.0:1789762967.040501:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762967.040502:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 682 00000100:00100000:1.0:1789762967.040512:0:3425:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789762967.040516:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074758784:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.040516:0:8056:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074758656:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762967.040524:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f5b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074760064:0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.040539:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 683 00002000:00000024:2.0:1789762967.040549:0:8056:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762967.040550:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074759680:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762967.040565:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074758784:12345-0@lo:400:kworker.0 Request processed in 54us (258us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762967.040573:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762967.040579:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 682 00000100:00100000:0.0:1789762967.040625:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762967.040714:0:8056:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3425:x1876702074758656:12345-0@lo:13:osp-pre-1-1.0 Request processed in 206us (516us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762967.040716:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3425:x1876702074759680:12345-0@lo:400:kworker.0 Request processed in 169us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762967.040725:0:8056:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 279 00000100:00100000:3.0:1789762967.040726:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 683 00000100:00100000:0.0:1789762967.040732:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074759808:0@lo:400:kworker.0 00000100:00100000:1.0:1789762967.040737:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3425:1876702074759680:0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.040747:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074759296 00000100:00100000:3.0:1789762967.040752:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074759296 00000100:00100000:0.0:1789762967.040764:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762967.040769:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074758656:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762967.040780:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762967.040792:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074759040 00000100:00100000:3.0:1789762967.040796:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074759040 00000100:00100000:0.0:1789762967.040812:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37fddd1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074758144:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762967.040815:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d252aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074759552:0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.040827:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 684 00000100:00100000:1.0:1789762967.040832:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074759168 00000100:00100000:1.0:1789762967.040838:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074759168 00000100:00100000:3.0:1789762967.040838:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11dbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074759296:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762967.040843:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d68700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074758784:0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.040851:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762967.040859:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1789762967.040868:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d6bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074758912:0@lo:400:kworker.0 00000100:00100000:1.0:1789762967.040886:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 451 00000100:00100000:2.0:1789762967.040887:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d69f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074758528:0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.040892:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11dbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3426:x1876702074759296:12345-0@lo:400:kworker.0 Request processed in 56us (265us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762967.040902:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 684 00000100:00100000:1.0:1789762967.040903:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074759168:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.040941:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074759552 00000100:00100000:0.0:1789762967.040942:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762967.040946:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 685 00000100:00100000:2.0:1789762967.040948:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074759552 00000100:00100000:3.0:1789762967.040960:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074759040:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762967.040981:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d3d6a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074759424:0@lo:400:kworker.0 00000100:00100000:1.0:1789762967.040981:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074759168:12345-0@lo:400:kworker.0 Request processed in 81us (677us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762967.040982:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 686 00000100:00100000:2.0:1789762967.041000:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074759552:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762967.041001:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 451 00000100:00100000:3.0:1789762967.041007:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074759040:12345-0@lo:400:kworker.0 Request processed in 50us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762967.041017:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 685 00000100:00100000:0.0:1789762967.041020:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762967.041026:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074758912 00000100:00100000:1.0:1789762967.041033:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074758912 00000100:00100000:3.0:1789762967.041038:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074759424 00000100:00100000:3.0:1789762967.041043:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074759424 00000100:00100000:0.0:1789762967.041046:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f7f58700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074759936:0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.041054:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074759552:12345-0@lo:400:kworker.0 Request processed in 58us (202us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762967.041063:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 686 00000100:00100000:1.0:1789762967.041079:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 452 00000100:00100000:3.0:1789762967.041080:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 687 00000100:00100000:0.0:1789762967.041089:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762967.041096:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074758912:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.041097:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074759424:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.041110:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d6b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074759040:0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.041126:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d252aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074759552:0@lo:400:kworker.0 00000100:00100000:0.0:1789762967.041133:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d69880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3426:1876702074759296:0@lo:400:kworker.0 00000100:00080000:2.0:1789762967.041156:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789762967.041162:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762967.041165:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074758912:12345-0@lo:400:kworker.0 Request processed in 71us (245us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762967.041167:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d6bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074758912:0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.041169:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3426:x1876702074759424:12345-0@lo:400:kworker.0 Request processed in 76us (152us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762967.041181:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762967.041187:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 452 00000100:00100000:0.0:1789762967.041190:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d6a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3426:1876702074759424:0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.041192:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 687 00000100:00100000:2.0:1789762967.041210:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074758400:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762967.041229:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074760192 00000100:00100000:1.0:1789762967.041234:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074760192 00000100:00100000:2.0:1789762967.041239:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762967.041251:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074758272 00000100:00100000:3.0:1789762967.041253:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074760064 00000100:00100000:0.0:1789762967.041256:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074758272 00000100:00100000:3.0:1789762967.041258:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074760064 00000100:00100000:1.0:1789762967.041279:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074759808 00000100:00100000:1.0:1789762967.041284:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074759808 00000100:00100000:2.0:1789762967.041285:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074758400 00000100:00100000:2.0:1789762967.041290:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074758400 00000100:00100000:3.0:1789762967.041290:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074759936 00000100:00100000:3.0:1789762967.041296:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074759936 00000100:00100000:0.0:1789762967.041300:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:1.0:1789762967.041312:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 279 00000100:00100000:0.0:1789762967.041316:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d252aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074758272:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762967.041326:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074760192:12345-0@lo:400:kworker.0 00002000:00000024:0.0:1789762967.041331:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762967.041334:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 281 00000100:00100000:3.0:1789762967.041334:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:2.0:1789762967.041356:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074758400:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762967.041356:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074760064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762967.041371:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074760192:12345-0@lo:400:kworker.0 Request processed in 49us (901us total) trans 0 rc 0/0 00002000:00000024:2.0:1789762967.041376:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762967.041380:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 279 00000100:00100000:0.0:1789762967.041386:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d252aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074758272:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (210us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762967.041394:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:1.0:1789762967.041405:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 281 00000100:00100000:3.0:1789762967.041411:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3425:x1876702074760064:12345-0@lo:400:kworker.0 Request processed in 64us (841us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762967.041418:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074759808:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.041425:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:2.0:1789762967.041426:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074758400:12345-0@lo:13:osp-pre-0-0.0 Request processed in 78us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762967.041434:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 281 00000100:00100000:0.0:1789762967.041438:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074758272:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762967.041451:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:3.0:1789762967.041462:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074759936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762967.041465:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074758400:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762967.041471:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074759808:12345-0@lo:400:kworker.0 Request processed in 58us (707us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762967.041478:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 281 00000100:00100000:3.0:1789762967.041505:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3426:x1876702074759936:12345-0@lo:400:kworker.0 Request processed in 45us (417us total) trans 0 rc 0/0 00000100:00080000:2.0:1789762967.041511:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762967.041512:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00080000:2.0:1789762967.041517:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762967.041537:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f58700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074759936:0@lo:400:kworker.0 00000100:00100000:1.0:1789762967.041550:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074760064:0@lo:400:kworker.0 00000100:00100000:0.0:1789762967.041557:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074759808:0@lo:400:kworker.0 00000100:00100000:1.0:1789762967.041565:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d3d68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074759168:0@lo:400:kworker.0 00000100:00100000:1.0:1789762967.041575:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f7f5ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074760192:0@lo:400:kworker.0 00010000:00010000:0.0:1789762967.168083:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762968.192114:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762969.088378:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36c11d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074760320:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762969.088442:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762969.088522:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074760320 00000100:00100000:1.0:1789762969.088527:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074760320 00000100:00100000:1.0:1789762969.088572:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:1.0:1789762969.088583:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074760320:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762969.088631:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762969.088679:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074760320:12345-0@lo:41:osp-pre-1-0.0 Request processed in 99us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762969.088686:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00100000:2.0:1789762969.088711:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36c11d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074760320:0@lo:41:osp-pre-1-0.0 00010000:00010000:2.0:1789762969.216036:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762969.409832:0:18564:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: uid, str: 500 00000100:00100000:3.0:1789762969.409840:0:18564:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: gid, str: 500 00000100:00100000:3.0:1789762969.409845:0:18564:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762969.410296:0:18564:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: uid, str: 500 00000100:00100000:3.0:1789762969.410300:0:18564:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: gid, str: 500 00000100:00100000:3.0:1789762969.410304:0:18564:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37d91d4a00 rate 5 gen 0 00000100:00100000:3.0:1789762969.410307:0:18564:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762969.410311:0:18564:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: uid, str: 500 00000100:00100000:3.0:1789762969.410313:0:18564:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: gid, str: 500 00000100:00100000:3.0:1789762969.410315:0:18564:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff9c37d91d4100 rate 5 gen 0 00000100:00100000:1.0:1789762969.421744:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762969.421775:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067555584 00000100:00100000:0.0:1789762969.421785:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067555584 00000100:00080000:1.0:1789762969.421824:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762969.421848:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 360 00000100:00100000:0.0:1789762969.421858:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:16251:x1876702067555584:12345-192.168.203.37@tcp:101:mkdir.0 00010000:00010000:0.0:1789762969.421874:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762969.421891:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1200/0xdf5a7ac59f17795b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9caee expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.422313:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177962 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.422327:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177962 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.422337:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177962 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762969.422377:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762969.422380:0:13389:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762969.422395:0:13389:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177962 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.422414:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1200/0xdf5a7ac59f17795b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caee expref: 7 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.422425:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177962 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caee expref: 6 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.422432:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762969.422435:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762969.422437:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d14f0a00, rc 0) 00000100:00100000:0.0:1789762969.422488:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:16251:x1876702067555584:12345-192.168.203.37@tcp:101:mkdir.0 Request processed in 633us (750us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762969.422494:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 360 00000100:00100000:2.0:1789762969.423899:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762969.423954:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067555712 00000100:00100000:1.0:1789762969.423960:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067555712 00000100:00100000:1.0:1789762969.423997:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 361 00000100:00100000:1.0:1789762969.424005:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067555712:12345-192.168.203.37@tcp:41:mkdir.0 00000004:00000024:1.0:1789762969.424028:0:6418:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:1.0:1789762969.424063:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067555712:12345-192.168.203.37@tcp:41:mkdir.0 Request processed in 60us (165us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762969.424069:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 361 00000100:00100000:0.0:1789762969.424127:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762969.424159:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067555968 00000100:00100000:1.0:1789762969.424163:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067555968 00000100:00100000:1.0:1789762969.424193:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 362 00000100:00100000:1.0:1789762969.424212:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:16251:x1876702067555968:12345-192.168.203.37@tcp:36:mkdir.0 00010000:00010000:1.0:1789762969.424239:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762969.424252:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177962 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caee expref: 6 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.424269:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762969.424273:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762969.424441:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177969 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.424450:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177969 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.424456:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177969 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.424469:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177970 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.424475:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177970 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.424480:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177970 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425723:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177977 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425731:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177977 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425737:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177977 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762969.425747:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007a75ad80 reply state 000000001585e2ed transno 4294967449 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177977 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425758:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177977 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425763:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:1.0:1789762969.425765:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1789762969.425767:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:1.0:1789762969.425770:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 16384 00010000:00010000:1.0:1789762969.425776:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177969 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425782:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177969 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425786:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177969 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425790:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177969 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425795:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81bc00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762969.425798:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177969 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425803:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177969 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425809:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762969.425810:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762969.425813:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177969 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425820:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177969 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762969.425828:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007a75ad80 reply state 000000001585e2ed transno 4294967449 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177970 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425833:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177970 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.425838:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743c00) 00010000:00010000:1.0:1789762969.425840:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1789762969.425841:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:1.0:1789762969.425880:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:16251:x1876702067555968:12345-192.168.203.37@tcp:36:mkdir.0 Request processed in 1669us (1755us total) trans 4294967449 rc 0/0 00000100:00100000:1.0:1789762969.425886:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 362 00000100:00100000:0.0:1789762969.431949:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762969.431991:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067556096 00000100:00100000:2.0:1789762969.431998:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067556096 00000100:00100000:2.0:1789762969.432031:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 363 00000100:00100000:2.0:1789762969.432043:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:16252:x1876702067556096:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:2.0:1789762969.432060:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762969.432072:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17797e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9caf5 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.432110:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177985 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.432122:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177985 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.432134:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177985 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762969.432153:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762969.432156:0:7744:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762969.432167:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177985 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.432181:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030800/0xdf5a7ac59f17797e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caf5 expref: 9 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.432196:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177985 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caf5 expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.432202:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762969.432203:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762969.432206:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c0030200, rc 0) 00000100:00100000:2.0:1789762969.432238:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:16252:x1876702067556096:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 200us (293us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762969.432243:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 363 00000100:00100000:2.0:1789762969.433347:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762969.433370:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067556352 00000100:00100000:3.0:1789762969.433376:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067556352 00000100:00100000:3.0:1789762969.433403:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 364 00000100:00100000:3.0:1789762969.433410:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:16252:x1876702067556352:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:3.0:1789762969.433418:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762969.433420:0:11692:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1789762969.433430:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177985 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9caf5 expref: 8 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433441:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762969.433445:0:11692:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762969.433451:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17798c lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cafc expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433631:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177993 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433643:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177993 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433653:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177993 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433665:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f17799a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433674:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f17799a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433681:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f17799a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433746:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779a1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433757:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779a1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433765:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779a1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762969.433774:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 16384 00010000:00010000:3.0:1789762969.433787:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779a1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433794:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779a1 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433801:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779a1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433806:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762969.433817:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779a1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433825:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779a1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433839:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:3.0:1789762969.433842:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762969.433846:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779a1 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433854:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779a1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433870:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177993 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433877:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177993 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433884:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177993 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433891:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177993 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433896:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caea00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762969.433899:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177993 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433904:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177993 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433913:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762969.433939:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762969.433944:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177993 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433955:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177993 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433966:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f17799a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433973:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f17799a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433979:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f17799a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433984:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762969.433988:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f17799a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.433995:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f17799a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.434003:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743c00) 00010000:00010000:3.0:1789762969.434006:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762969.434009:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f17799a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.434016:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f17799a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.434040:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17798c lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cafc expref: 8 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.434057:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762969.434060:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762969.434064:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f17798c lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cafc expref: 8 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.434075:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8cae400, rc 0) 00000100:00100000:3.0:1789762969.434131:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:16252:x1876702067556352:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 719us (782us total) trans 0 rc 301/301 00000100:00100000:3.0:1789762969.434139:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 364 00000100:00100000:0.0:1789762969.435739:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762969.435778:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067556480 00000100:00100000:1.0:1789762969.435785:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067556480 00000100:00100000:1.0:1789762969.435825:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 365 00000100:00100000:1.0:1789762969.435835:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11db100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:16252:x1876702067556480:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:1.0:1789762969.435845:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762969.435857:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81aa00/0xdf5a7ac59f1779a8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb03 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.435895:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f1779af lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.435904:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f1779af lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.435949:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f1779af lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762969.435972:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762969.435977:0:6420:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:1.0:1789762969.435993:0:6420:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f1779af lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.436013:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81aa00/0xdf5a7ac59f1779a8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb03 expref: 9 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.436023:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f1779af lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb03 expref: 8 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.436040:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762969.436042:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762969.436044:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee81a400, rc 0) 00000100:00100000:1.0:1789762969.436079:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11db100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:16252:x1876702067556480:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 247us (344us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762969.436083:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 365 00000100:00100000:1.0:1789762969.437161:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762969.437192:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067556736 00000100:00100000:0.0:1789762969.437197:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067556736 00000100:00100000:0.0:1789762969.437229:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 366 00000100:00100000:0.0:1789762969.437236:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:16252:x1876702067556736:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762969.437244:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762969.437253:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1600/0xdf5a7ac59f1779b6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb0a expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437296:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1779bd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437306:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1779bd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437316:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1779bd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437338:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1779c4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437347:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1779c4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437355:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1779c4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762969.437433:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 16384 00010000:00010000:0.0:1789762969.437496:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1779bd lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437503:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1779bd lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437507:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1779bd lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437512:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1779bd lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437516:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f1a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762969.437518:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1779bd lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437523:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1779bd lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437529:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762969.437532:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762969.437534:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1779bd lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437538:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f1779bd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437545:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1779c4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437550:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1779c4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437554:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1779c4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437558:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f0400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762969.437560:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1779c4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437567:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1779c4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437572:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743c00) 00010000:00010000:0.0:1789762969.437574:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762969.437576:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1779c4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437579:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f1779c4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437615:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1600/0xdf5a7ac59f1779b6 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb0a expref: 10 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437622:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762969.437623:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762969.437626:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1600/0xdf5a7ac59f1779b6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb0a expref: 10 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.437631:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d14f1600, rc 0) 00000100:00100000:0.0:1789762969.437661:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:16252:x1876702067556736:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 426us (500us total) trans 4294967450 rc 301/301 00000100:00100000:0.0:1789762969.437666:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 366 00000100:00100000:2.0:1789762969.439211:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762969.439246:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067556992 00000100:00100000:1.0:1789762969.439252:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067556992 00000100:00100000:1.0:1789762969.439298:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 367 00000100:00100000:1.0:1789762969.439305:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:16252:x1876702067556992:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:1.0:1789762969.439351:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f1779d2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.439364:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f1779d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.439371:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f1779d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762969.439494:0:6418:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000004d039d9 reply state 0000000074191faa transno 4294967451 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f1779d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.439507:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f1779d2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.439515:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:1.0:1789762969.439518:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:1.0:1789762969.439521:0:6418:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:1.0:1789762969.439527:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 16384 00000100:00100000:1.0:1789762969.439578:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:16252:x1876702067556992:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 273us (371us total) trans 4294967451 rc 0/0 00000100:00100000:1.0:1789762969.439584:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 367 00000100:00100000:2.0:1789762969.440503:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762969.440573:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067557248 00000100:00100000:0.0:1789762969.440579:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067557248 00000100:00100000:0.0:1789762969.440624:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 107 00000100:00100000:0.0:1789762969.440633:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:16252:x1876702067557248:12345-192.168.203.37@tcp:35:lfs.0 00000100:00100000:0.0:1789762969.440759:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:16252:x1876702067557248:12345-192.168.203.37@tcp:35:lfs.0 Request processed in 127us (257us total) trans 4294967452 rc 0/0 00000100:00100000:0.0:1789762969.440775:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 107 00000100:00100000:0.0:1789762969.445826:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762969.445865:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067557376 00000100:00100000:3.0:1789762969.445872:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067557376 00000100:00100000:3.0:1789762969.445904:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 368 00000100:00100000:3.0:1789762969.445912:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:16253:x1876702067557376:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:3.0:1789762969.445951:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762969.445961:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1779d9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb11 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.445993:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1779e0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446002:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1779e0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446020:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1779e0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446027:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f1779e7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446033:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f1779e7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446038:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f1779e7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446081:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779ee lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446087:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779ee lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446092:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779ee lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762969.446107:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 16384 00000004:00200000:3.0:1789762969.446110:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762969.446120:0:6419:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779ee lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446127:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1779e0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446133:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1779e0 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446137:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1779e0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446141:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1779e0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446149:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762969.446152:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1779e0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446157:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1779e0 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446163:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762969.446165:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762969.446168:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1779e0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446171:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f1779e0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446181:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f1779e7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446185:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f1779e7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446190:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f1779e7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446194:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caea00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762969.446196:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f1779e7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446200:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f1779e7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446208:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743c00) 00010000:00010000:3.0:1789762969.446210:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762969.446211:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f1779e7 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446215:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f1779e7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446231:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f1779d9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb11 expref: 14 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446241:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779ee lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb11 expref: 13 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.446245:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:3.0:1789762969.446247:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762969.446248:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8caf800, rc 0) 00000100:00100000:3.0:1789762969.446281:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:16253:x1876702067557376:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 370us (456us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762969.446287:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 368 00000100:00100000:2.0:1789762969.447343:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762969.447375:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067557632 00000100:00100000:3.0:1789762969.447380:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067557632 00000100:00100000:3.0:1789762969.447410:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 369 00000100:00100000:3.0:1789762969.447417:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:16253:x1876702067557632:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:3.0:1789762969.447436:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762969.447447:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779ee lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb11 expref: 13 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.447460:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:3.0:1789762969.447462:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762969.447471:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f1779f5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.447478:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f1779f5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.447483:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f1779f5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762969.447576:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000444b8ea5 reply state 00000000b1a73df0 transno 4294967453 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f1779f5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.447584:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f1779f5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.447606:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:3.0:1789762969.447608:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:3.0:1789762969.447610:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762969.447614:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 16384 00000100:00100000:3.0:1789762969.447658:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:16253:x1876702067557632:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 243us (316us total) trans 4294967453 rc 0/0 00000100:00100000:3.0:1789762969.447664:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 369 00000100:00100000:2.0:1789762969.464432:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762969.464480:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067557760 00000100:00100000:3.0:1789762969.464490:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067557760 00000100:00100000:3.0:1789762969.464541:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 370 00000100:00100000:3.0:1789762969.464549:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:16257:x1876702067557760:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762969.464569:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762969.464586:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779fc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb18 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464664:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177a03 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464675:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177a03 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464686:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177a03 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464708:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177a0a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464717:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177a0a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464726:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177a0a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464833:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177a11 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464843:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177a11 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464852:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177a11 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762969.464891:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 16384 00000004:00200000:3.0:1789762969.464894:0:11692:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762969.464908:0:11692:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177a11 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464940:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177a03 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464950:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177a03 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464958:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177a03 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464966:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177a03 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464973:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762969.464983:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177a03 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.464992:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177a03 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.465002:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762969.465005:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762969.465008:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177a03 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.465013:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177a03 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.465033:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177a0a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.465038:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177a0a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.465043:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177a0a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.465047:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caea00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762969.465048:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177a0a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.465052:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177a0a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.465056:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743c00) 00010000:00010000:3.0:1789762969.465057:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762969.465059:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177a0a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.465063:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177a0a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.465080:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f1779fc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb18 expref: 16 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.465089:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177a11 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb18 expref: 15 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.465093:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:3.0:1789762969.465095:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789762969.465096:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8cae000, rc 0) 00000100:00100000:3.0:1789762969.465155:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:16257:x1876702067557760:12345-192.168.203.37@tcp:101:dd.500 Request processed in 605us (729us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762969.465169:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 370 00000100:00100000:1.0:1789762969.466339:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762969.466380:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067558016 00000100:00100000:0.0:1789762969.466386:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067558016 00000100:00100000:0.0:1789762969.466425:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 371 00000100:00100000:0.0:1789762969.466435:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:16257:x1876702067558016:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:0.0:1789762969.466446:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762969.466457:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f177a18 lrc: 2/0,0 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb1f expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.466540:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f177a1f lrc: 3/0,1 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.466552:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f177a1f lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.466562:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f177a1f lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.466751:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177a26 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.466771:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177a26 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.466781:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177a26 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762969.467582:0:6420:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x9:0x0], index 0: 0 00000004:00080000:0.0:1789762969.467609:0:6420:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x2:0x0], index 1: 0 00010000:00010000:0.0:1789762969.467779:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1400/0xdf5a7ac59f177a2d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467787:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1400/0xdf5a7ac59f177a2d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467792:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1400/0xdf5a7ac59f177a2d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467821:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f177a1f lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467827:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f177a1f lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467832:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f177a1f lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467836:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f177a1f lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467840:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f1a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762969.467843:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f177a1f lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467848:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f177a1f lrc: 5/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467856:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:0.0:1789762969.467859:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762969.467863:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f177a1f lrc: 3/0,0 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467867:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1a00/0xdf5a7ac59f177a1f lrc: 0/0,0 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762969.467876:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006e38b2ba reply state 000000005e622184 transno 4294967454 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177a26 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467885:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177a26 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467889:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x73550ecb (ffff9c37e37a7100) 00010000:00010000:0.0:1789762969.467891:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762969.467893:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762969.467943:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0400/0xdf5a7ac59f177a18 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb1f expref: 18 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467962:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1400/0xdf5a7ac59f177a2d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb1f expref: 17 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.467968:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9c37e37a7e00) 00010000:00010000:0.0:1789762969.467970:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762969.467972:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d14f1400, rc 0) 00000100:00100000:0.0:1789762969.468030:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:16257:x1876702067558016:12345-192.168.203.37@tcp:101:dd.500 Request processed in 1598us (1693us total) trans 4294967454 rc 0/0 00000100:00100000:0.0:1789762969.468037:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 371 00000100:00100000:2.0:1789762969.470574:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762969.470617:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067558144 00000100:00100000:3.0:1789762969.470623:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067558144 00000100:00100000:3.0:1789762969.470683:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 205 00000100:00080000:1.0:1789762969.470685:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762969.470696:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:16257:x1876702067558144:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762969.470707:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762969.470721:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177a3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb26 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762969.470745:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x9:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762969.470759:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9:0x0].0x0 since=442s v=(0 0) 00010000:00010000:3.0:1789762969.470762:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762969.470769:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177a3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb26 expref: 6 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.470781:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177a3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb26 expref: 6 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.470791:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8caf200, rc 0) 00000100:00100000:3.0:1789762969.470830:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:16257:x1876702067558144:12345-192.168.203.37@tcp:101:dd.500 Request processed in 137us (257us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762969.470842:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 205 00000100:00100000:2.0:1789762969.479216:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762969.479252:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067558272 00000100:00100000:1.0:1789762969.479259:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067558272 00002000:00100000:1.0:1789762969.479294:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x9:0x0] (0->1048575) req@ffff9c36cb451880 x1876702067558272/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/0 e 0 to 0 dl 1789762980 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762969.479302:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb451880 with x1876702067558272 ext(0->1048575) 00002000:00010000:1.0:1789762969.479306:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36cb451880 00000100:00100000:1.0:1789762969.479319:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376761845345 time 376761907751 state 0x0 ref 1 00000100:00100000:1.0:1789762969.479338:0:8059:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 9: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 376761845345 time 376761907751 state 0x0 ref 1 00000100:00100000:1.0:1789762969.479349:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 3 deadline 442926426217 time 442726426217 state 0x1 ref 1 00000100:00100000:1.0:1789762969.479371:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 2 deadline 442926426217 time 442726449142 state 0x0 ref 1 00000100:00100000:1.0:1789762969.479381:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb451880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2397:x1876702067558272:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762969.484997:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb451880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067558272:12345-192.168.203.37@tcp:4:dd.500 Request processed in 5615us (5779us total) trans 4294968133 rc 0/0 00000100:00100000:3.0:1789762969.485006:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1124 00000100:00100000:3.0:1789762969.485014:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 2 deadline 442926426217 time 442726449142 state 0x2 ref 0 00002000:00100000:3.0:1789762969.485023:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x9:0x0] (0->1048575) req@ffff9c36cb451880 x1876702067558272/t4294968133(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/448 e 0 to 0 dl 1789762980 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762969.485030:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb451880 with x1876702067558272 ext(0->1048575) 00002000:00010000:3.0:1789762969.485034:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36cb451880 00000100:00100000:1.0:1789762969.489067:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762969.489106:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067558400 00000100:00100000:0.0:1789762969.489116:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067558400 00002000:00100000:0.0:1789762969.489158:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x9:0x0] (1048576->2097151) req@ffff9c37fddd1180 x1876702067558400/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/0 e 0 to 0 dl 1789762980 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762969.489170:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fddd1180 with x1876702067558400 ext(1048576->2097151) 00002000:00010000:0.0:1789762969.489176:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fddd1180 00000100:00100000:0.0:1789762969.489196:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 2 deadline 442926426217 time 442726449142 state 0x0 ref 1 00000100:00100000:0.0:1789762969.489211:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 2 deadline 442926449142 time 442726449142 state 0x1 ref 1 00000100:00100000:0.0:1789762969.489249:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 1 deadline 442926449142 time 442736323411 state 0x0 ref 1 00000100:00100000:0.0:1789762969.489264:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fddd1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2398:x1876702067558400:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762969.492032:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fddd1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2398:x1876702067558400:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2769us (2965us total) trans 4294968134 rc 0/0 00000100:00100000:0.0:1789762969.492040:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1125 00000100:00100000:0.0:1789762969.492048:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 1 deadline 442926449142 time 442736323411 state 0x2 ref 0 00002000:00100000:0.0:1789762969.492057:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x9:0x0] (1048576->2097151) req@ffff9c37fddd1180 x1876702067558400/t4294968134(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/448 e 0 to 0 dl 1789762980 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762969.492065:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37fddd1180 with x1876702067558400 ext(1048576->2097151) 00002000:00010000:0.0:1789762969.492068:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37fddd1180 00000100:00100000:1.0:1789762969.499692:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762969.499763:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067558528 00000100:00100000:3.0:1789762969.499771:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067558528 00002000:00100000:3.0:1789762969.499808:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x9:0x0] (2097152->3145727) req@ffff9c36cb591f80 x1876702067558528/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/0 e 0 to 0 dl 1789762980 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762969.499820:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb591f80 with x1876702067558528 ext(2097152->3145727) 00002000:00010000:3.0:1789762969.499826:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36cb591f80 00000100:00100000:3.0:1789762969.499840:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 1 deadline 442926449142 time 442736323411 state 0x0 ref 1 00000100:00100000:3.0:1789762969.499854:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 1 deadline 442936323411 time 442736323411 state 0x1 ref 1 00000100:00100000:3.0:1789762969.499887:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 442936323411 time 442746962700 state 0x0 ref 1 00000100:00100000:3.0:1789762969.499903:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb591f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2398:x1876702067558528:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762969.502708:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb591f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:2398:x1876702067558528:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2808us (3016us total) trans 4294968135 rc 0/0 00000100:00100000:3.0:1789762969.502718:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1126 00000100:00100000:3.0:1789762969.502728:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 442936323411 time 442746962700 state 0x2 ref 0 00002000:00100000:3.0:1789762969.502744:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x9:0x0] (2097152->3145727) req@ffff9c36cb591f80 x1876702067558528/t4294968135(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/448 e 0 to 0 dl 1789762980 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762969.502755:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb591f80 with x1876702067558528 ext(2097152->3145727) 00002000:00010000:3.0:1789762969.502760:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36cb591f80 00000100:00100000:2.0:1789762969.506485:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762969.506603:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067558656 00000100:00100000:3.0:1789762969.506609:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067558656 00000100:00100000:1.0:1789762969.506649:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00002000:00100000:3.0:1789762969.506677:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x9:0x0] (3145728->4194303) req@ffff9c36cb451500 x1876702067558656/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/0 e 0 to 0 dl 1789762980 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762969.506690:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb451500 with x1876702067558656 ext(3145728->4194303) 00002000:00010000:3.0:1789762969.506694:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36cb451500 00000100:00100000:2.0:1789762969.506696:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067558784 00000100:00100000:2.0:1789762969.506701:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067558784 00000100:00100000:3.0:1789762969.506709:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 442936323411 time 442746962700 state 0x0 ref 1 00000100:00100000:3.0:1789762969.506723:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 442946962700 time 442746962700 state 0x1 ref 1 00000100:00100000:2.0:1789762969.506731:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 206 00000100:00100000:2.0:1789762969.506737:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb592a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0+-99:2394:x1876702067558784:12345-192.168.203.37@tcp:8:dd.500 00010000:00080000:2.0:1789762969.506761:0:9038:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp t0 exp 0000000000000000 cur 442 last 0 00000020:00000080:2.0:1789762969.506784:0:9038:0:(genops.c:1385:class_connect()) connect: client fa6fc127-0d58-47ca-82d1-2c8ad7b086c1, cookie 0xdf5a7ac59f177a42 00002000:00100000:2.0:1789762969.506801:0:9038:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli fa6fc127-0d58-47ca-82d1-2c8ad7b086c1/ffff9c36d1b3c000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:2.0:1789762969.506817:0:9038:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:2.0:1789762969.506985:0:9038:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:2.0:1789762969.507009:0:9038:0:(import.c:65:import_set_state_nolock()) ffff9c36d1b3b000 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789762969.507044:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb592a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:2394:x1876702067558784:12345-192.168.203.37@tcp:8:dd.500 Request processed in 306us (397us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762969.507050:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 206 00000100:00100000:2.0:1789762969.507759:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762969.507806:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067558912 00000100:00100000:3.0:1789762969.507812:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067558912 00000100:00080000:1.0:1789762969.507859:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762969.507864:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 207 00000100:00100000:3.0:1789762969.507876:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb451c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:16257:x1876702067558912:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:3.0:1789762969.507887:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762969.507906:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a558e1d5/0xdf5a7ac59f177a49 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb2d expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762969.507960:0:9037:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x2:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762969.507976:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x2:0x0].0x0 since=442s v=(0 0) 00010000:00010000:3.0:1789762969.507982:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762969.507992:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a558e1d5/0xdf5a7ac59f177a49 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb2d expref: 7 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.508018:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a558e1d5/0xdf5a7ac59f177a49 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb2d expref: 7 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.508032:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8caf800, rc 0) 00000100:00100000:2.0:1789762969.508112:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb451c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:16257:x1876702067558912:12345-192.168.203.37@tcp:101:dd.500 Request processed in 240us (351us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762969.508118:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 207 00000100:00100000:0.0:1789762969.517571:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762969.517656:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067559040 00000100:00100000:3.0:1789762969.517666:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067559040 00002000:00100000:3.0:1789762969.517704:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x2:0x0] (0->1048575) req@ffff9c36c11d9880 x1876702067559040/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/0 e 0 to 0 dl 1789762980 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762969.517718:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36c11d9880 with x1876702067559040 ext(0->1048575) 00002000:00010000:3.0:1789762969.517724:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36c11d9880 00000100:00100000:3.0:1789762969.517742:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 442946962700 time 442746962700 state 0x1 ref 2 00000100:00100000:2.0:1789762969.524645:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762969.524708:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067559168 00000100:00100000:0.0:1789762969.524718:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067559168 00002000:00100000:0.0:1789762969.524759:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2:0x0] (1048576->2097151) req@ffff9c36cb452d80 x1876702067559168/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/0 e 0 to 0 dl 1789762980 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762969.524769:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb452d80 with x1876702067559168 ext(1048576->2097151) 00002000:00010000:0.0:1789762969.524773:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36cb452d80 00000100:00100000:0.0:1789762969.524787:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 442946962700 time 442746962700 state 0x1 ref 3 00000001:00080000:3.0:1789762969.610226:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967454 is committed 00010000:00010000:0.0:1789762969.610251:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177970 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610266:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177970 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610273:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177970 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610279:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81ac00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762969.610283:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177970 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610289:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177970 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610307:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743c00) 00010000:00010000:0.0:1789762969.610310:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762969.610314:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177970 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610319:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177970 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610336:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177977 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610342:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177977 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610346:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177977 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610349:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81ba00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762969.610351:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177977 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610355:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177977 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610359:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:0.0:1789762969.610360:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762969.610362:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177977 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610366:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177977 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610376:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f1779d2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610380:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f1779d2 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610383:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f1779d2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610386:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81a000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762969.610389:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f1779d2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610394:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f1779d2 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610398:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:0.0:1789762969.610399:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789762969.610401:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f1779d2 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610404:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f1779d2 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610410:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f1779f5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610414:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f1779f5 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610417:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f1779f5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610420:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caee00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762969.610422:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f1779f5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610426:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f1779f5 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610430:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:0.0:1789762969.610432:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762969.610434:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f1779f5 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610438:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caee00/0xdf5a7ac59f1779f5 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610446:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177a26 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610450:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177a26 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610454:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177a26 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610457:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d14f0a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762969.610459:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177a26 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610462:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177a26 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610467:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x73550ecb (ffff9c37e37a7100) 00010000:00010000:0.0:1789762969.610468:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762969.610470:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177a26 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.610473:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f0a00/0xdf5a7ac59f177a26 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762969.699964:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443147028792 time 442947028792 state 0x1 ref 3 00000100:00100000:3.0:1789762969.699991:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb451500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:2397:x1876702067558656:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762969.703022:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb451500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:2397:x1876702067558656:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3042us (196537us total) trans 4294968136 rc 0/0 00000100:00100000:3.0:1789762969.703029:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1127 00002000:00100000:3.0:1789762969.703040:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x9:0x0] (3145728->4194303) req@ffff9c36cb451500 x1876702067558656/t4294968136(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/448 e 0 to 0 dl 1789762980 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762969.703053:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb451500 with x1876702067558656 ext(3145728->4194303) 00002000:00010000:3.0:1789762969.703059:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36cb451500 00000100:00100000:3.0:1789762969.900020:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443347084737 time 443147084737 state 0x1 ref 2 00000100:00100000:3.0:1789762969.900043:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067559040:12345-192.168.203.37@tcp:4:dd.500 00000001:00100000:0.0:1789762969.914639:0:9000:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of fa6fc127-0d58-47ca-82d1-2c8ad7b086c1 00000001:00080000:0.0:1789762969.924867:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967418 is committed 00000100:00100000:3.0:1789762969.924974:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067559040:12345-192.168.203.37@tcp:4:dd.500 Request processed in 24935us (407404us total) trans 4294967418 rc 0/0 00000100:00100000:3.0:1789762969.924983:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1128 00002000:00100000:3.0:1789762969.924994:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x2:0x0] (0->1048575) req@ffff9c36c11d9880 x1876702067559040/t4294967418(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/448 e 0 to 0 dl 1789762980 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762969.925003:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36c11d9880 with x1876702067559040 ext(0->1048575) 00002000:00010000:3.0:1789762969.925008:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36c11d9880 00000100:00100000:1.0:1789762969.927303:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762969.927377:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067559296 00000100:00100000:0.0:1789762969.927384:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067559296 00002000:00100000:0.0:1789762969.927414:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2:0x0] (2097152->3145727) req@ffff9c36cb590380 x1876702067559296/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/0 e 0 to 0 dl 1789762980 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762969.927422:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb590380 with x1876702067559296 ext(2097152->3145727) 00002000:00010000:0.0:1789762969.927425:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36cb590380 00000100:00100000:0.0:1789762969.927441:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443347084737 time 443147084737 state 0x1 ref 2 00000100:00100000:0.0:1789762969.937258:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762969.937333:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067559424 00000100:00100000:1.0:1789762969.937338:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067559424 00002000:00100000:1.0:1789762969.937358:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x2:0x0] (3145728->4194303) req@ffff9c36c11d8700 x1876702067559424/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/0 e 0 to 0 dl 1789762980 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762969.937366:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36c11d8700 with x1876702067559424 ext(3145728->4194303) 00002000:00010000:1.0:1789762969.937369:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36c11d8700 00000100:00100000:1.0:1789762969.937377:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443347084737 time 443147084737 state 0x1 ref 3 00000100:00100000:2.0:1789762969.944152:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762969.944232:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067559552 00000100:00100000:1.0:1789762969.944238:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067559552 00002000:00100000:1.0:1789762969.944263:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x9:0x0] (4194304->5242879) req@ffff9c36cb450700 x1876702067559552/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/0 e 0 to 0 dl 1789762980 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762969.944278:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb450700 with x1876702067559552 ext(4194304->5242879) 00002000:00010000:1.0:1789762969.944281:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36cb450700 00000100:00100000:1.0:1789762969.944291:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443347084737 time 443147084737 state 0x1 ref 4 00000100:00100000:1.0:1789762969.954057:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762969.954100:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067559680 00000100:00100000:3.0:1789762969.954108:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067559680 00002000:00100000:3.0:1789762969.954133:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x9:0x0] (5242880->6291455) req@ffff9c36cb592d80 x1876702067559680/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/0 e 0 to 0 dl 1789762980 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762969.954144:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb592d80 with x1876702067559680 ext(5242880->6291455) 00002000:00010000:3.0:1789762969.954148:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36cb592d80 00000100:00100000:3.0:1789762969.954160:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443347084737 time 443147084737 state 0x1 ref 5 00000100:00100000:0.0:1789762969.960462:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762969.960513:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067559808 00000100:00100000:3.0:1789762969.960520:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067559808 00002000:00100000:3.0:1789762969.960544:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x9:0x0] (6291456->7340031) req@ffff9c37f7f5ad80 x1876702067559808/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/0 e 0 to 0 dl 1789762980 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762969.960555:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f7f5ad80 with x1876702067559808 ext(6291456->7340031) 00002000:00010000:3.0:1789762969.960559:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37f7f5ad80 00000100:00100000:3.0:1789762969.960582:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443347084737 time 443147084737 state 0x1 ref 6 00000100:00100000:2.0:1789762969.967190:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762969.967236:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067559936 00000100:00100000:0.0:1789762969.967241:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067559936 00002000:00100000:0.0:1789762969.967262:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x9:0x0] (7340032->8388607) req@ffff9c36cb453800 x1876702067559936/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/0 e 0 to 0 dl 1789762980 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762969.967270:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb453800 with x1876702067559936 ext(7340032->8388607) 00002000:00010000:0.0:1789762969.967273:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36cb453800 00000100:00100000:0.0:1789762969.967282:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443347084737 time 443147084737 state 0x1 ref 7 00000100:00100000:3.0:1789762970.100102:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443547167357 time 443347167357 state 0x1 ref 7 00000100:00100000:3.0:1789762970.100131:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb452d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067559168:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762970.103463:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb452d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:2397:x1876702067559168:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3344us (578818us total) trans 4294967419 rc 0/0 00000100:00100000:3.0:1789762970.103470:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1129 00002000:00100000:3.0:1789762970.103480:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2:0x0] (1048576->2097151) req@ffff9c36cb452d80 x1876702067559168/t4294967419(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/448 e 0 to 0 dl 1789762980 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762970.103488:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb452d80 with x1876702067559168 ext(1048576->2097151) 00002000:00010000:3.0:1789762970.103493:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36cb452d80 00000100:00100000:0.0:1789762970.106138:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.106180:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067560064 00000100:00100000:3.0:1789762970.106187:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067560064 00002000:00100000:3.0:1789762970.106213:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x2:0x0] (4194304->5242879) req@ffff9c37f7f5b800 x1876702067560064/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762970.106221:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f7f5b800 with x1876702067560064 ext(4194304->5242879) 00002000:00010000:3.0:1789762970.106224:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f7f5b800 00000100:00100000:3.0:1789762970.106236:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443547167357 time 443347167357 state 0x1 ref 7 00000100:00100000:2.0:1789762970.112036:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762970.112053:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074760448:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789762970.112092:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762970.112131:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074760448 00000100:00100000:0.0:1789762970.112137:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074760448 00000100:00100000:0.0:1789762970.112179:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:0.0:1789762970.112187:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb451f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074760448:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789762970.112221:0:13302:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:0.0:1789762970.112273:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb451f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3426:x1876702074760448:12345-0@lo:41:osp-pre-0-1.0 Request processed in 87us (179us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762970.112282:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:2.0:1789762970.112306:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3426:1876702074760448:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762970.114674:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.114715:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067560192 00000100:00100000:3.0:1789762970.114720:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067560192 00002000:00100000:3.0:1789762970.114739:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x2:0x0] (5242880->6291455) req@ffff9c37d184c700 x1876702067560192/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762970.114747:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184c700 with x1876702067560192 ext(5242880->6291455) 00002000:00010000:3.0:1789762970.114749:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184c700 00000100:00100000:3.0:1789762970.114759:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443547167357 time 443347167357 state 0x1 ref 8 00010000:00010000:0.0:1789762970.240119:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762970.300234:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443747287488 time 443547287488 state 0x1 ref 8 00000100:00100000:0.0:1789762970.300264:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:2398:x1876702067559296:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:0.0:1789762970.303823:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:2398:x1876702067559296:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3572us (376521us total) trans 4294967420 rc 0/0 00000100:00100000:0.0:1789762970.303831:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1130 00002000:00100000:0.0:1789762970.303842:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2:0x0] (2097152->3145727) req@ffff9c36cb590380 x1876702067559296/t4294967420(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/448 e 0 to 0 dl 1789762980 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762970.303858:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb590380 with x1876702067559296 ext(2097152->3145727) 00002000:00010000:0.0:1789762970.303865:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36cb590380 00000100:00100000:1.0:1789762970.306736:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.306782:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067560320 00000100:00100000:3.0:1789762970.306791:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067560320 00002000:00100000:3.0:1789762970.306819:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2:0x0] (6291456->7340031) req@ffff9c37d184d180 x1876702067560320/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762970.306826:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184d180 with x1876702067560320 ext(6291456->7340031) 00002000:00010000:3.0:1789762970.306829:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184d180 00000100:00100000:3.0:1789762970.306842:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443747287488 time 443547287488 state 0x1 ref 8 00000100:00100000:2.0:1789762970.314145:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762970.314172:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.314225:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067560448 00000100:00100000:3.0:1789762970.314231:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067560448 00000100:00100000:1.0:1789762970.314232:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067560704 00000100:00100000:1.0:1789762970.314238:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067560704 00002000:00100000:3.0:1789762970.314255:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x2:0x0] (7340032->8388607) req@ffff9c36cb452300 x1876702067560448/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762970.314264:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb452300 with x1876702067560448 ext(7340032->8388607) 00002000:00010000:3.0:1789762970.314267:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36cb452300 00000100:00100000:1.0:1789762970.314279:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 108 00000100:00100000:3.0:1789762970.314283:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443747287488 time 443547287488 state 0x1 ref 9 00000100:00100000:1.0:1789762970.314294:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb452680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:16257:x1876702067560704:12345-192.168.203.37@tcp:35:dd.500 00000001:00080000:1.0:1789762970.314549:0:6422:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967455 00000100:00100000:1.0:1789762970.314672:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb452680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:16257:x1876702067560704:12345-192.168.203.37@tcp:35:dd.500 Request processed in 381us (497us total) trans 4294967456 rc 0/0 00000100:00100000:1.0:1789762970.314682:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 108 00000020:00080000:1.0:1789762970.314688:0:6422:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1789762970.365754:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762970.365822:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067560960 00000100:00100000:2.0:1789762970.365831:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067560960 00000100:00100000:2.0:1789762970.365881:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 372 00000100:00100000:2.0:1789762970.365890:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:16268:x1876702067560960:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762970.365913:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762970.365953:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031400/0xdf5a7ac59f177a50 lrc: 2/0,0 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb34 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.366188:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177a57 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.366200:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177a57 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.366210:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177a57 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.366277:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177a5e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.366284:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177a5e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.366289:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177a5e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762970.366841:0:13389:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x3:0x0], index 1: 0 00000004:00080000:2.0:1789762970.366849:0:13389:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0xa:0x0], index 0: 0 00010000:00010000:2.0:1789762970.367044:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177a65 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367055:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177a65 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367064:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177a65 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367100:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177a57 lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367107:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177a57 lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367112:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177a57 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367117:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177a57 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367121:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762970.367124:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177a57 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367129:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177a57 lrc: 5/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367138:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:2.0:1789762970.367141:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762970.367144:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177a57 lrc: 3/0,0 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367149:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177a57 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762970.367159:0:13389:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000623ffbc8 reply state 000000001bed898d transno 4294967457 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177a5e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367167:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177a5e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367171:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x2b0e463d (ffff9c37d3bee800) 00010000:00010000:2.0:1789762970.367173:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1789762970.367175:0:13389:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762970.367195:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0031400/0xdf5a7ac59f177a50 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb34 expref: 13 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367206:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177a65 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb34 expref: 12 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.367213:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9c37d3beea00) 00010000:00010000:2.0:1789762970.367216:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762970.367219:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c0030c00, rc 0) 00000100:00100000:2.0:1789762970.367281:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:16268:x1876702067560960:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1391us (1533us total) trans 4294967457 rc 0/0 00000100:00100000:2.0:1789762970.367289:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 372 00000100:00100000:2.0:1789762970.374958:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.375005:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067561088 00000100:00100000:3.0:1789762970.375013:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067561088 00000100:00100000:3.0:1789762970.375058:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 208 00000100:00100000:3.0:1789762970.375069:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb450000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+15:16268:x1876702067561088:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762970.375082:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762970.375098:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb3b expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762970.375124:0:8053:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x3:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762970.375151:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=443s v=(0 0) 00010000:00010000:3.0:1789762970.375156:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762970.375164:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 16 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.375175:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 16 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.375184:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8caea00, rc 0) 00000100:00100000:3.0:1789762970.375268:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb450000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+16:16268:x1876702067561088:12345-192.168.203.37@tcp:101:dd.0 Request processed in 152us (262us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762970.375279:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 208 00000100:00100000:2.0:1789762970.375327:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762970.375358:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067561216 00000100:00100000:1.0:1789762970.375364:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067561216 00000100:00100000:1.0:1789762970.375392:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 209 00000100:00100000:1.0:1789762970.375400:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb453100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+16:16269:x1876702067561216:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762970.375408:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762970.375416:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cc930a86/0xdf5a7ac59f177a7a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb42 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.375437:0:9038:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 17 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.375444:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762970.375447:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789762970.375450:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cc930a86/0xdf5a7ac59f177a7a lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb42 expref: 17 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.375479:0:9038:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 17 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.375491:0:9038:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 17 pid: 8053 timeout: 543 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762970.375502:0:9038:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d184c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:9038:1876702074760576:192.168.203.37@tcp:104: 00000100:00100000:1.0:1789762970.375559:0:9038:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252f3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762970.376790:0:9038:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d184c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:9038:1876702074760576:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762970.376806:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cc930a86/0xdf5a7ac59f177a7a lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb42 expref: 17 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.376815:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee81bc00, rc 0) 00000100:00100000:2.0:1789762970.376841:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb453100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:16269:x1876702067561216:12345-192.168.203.37@tcp:101:dd.0 Request processed in 1442us (1515us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762970.376849:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 209 00000100:00100000:2.0:1789762970.377113:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762970.377155:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067561344 00000100:00100000:1.0:1789762970.377160:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067561344 00000100:00100000:1.0:1789762970.377187:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 210 00000100:00100000:1.0:1789762970.377194:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:16272:x1876702067561344:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762970.377201:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762970.377213:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177a81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb49 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762970.377232:0:9037:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0xa:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789762970.377244:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0xa:0x0].0x0 since=443s v=(0 0) 00010000:00010000:1.0:1789762970.377248:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0xa:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762970.377255:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177a81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb49 expref: 19 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.377268:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177a81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb49 expref: 19 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.377276:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee81a800, rc 0) 00000100:00100000:1.0:1789762970.377302:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:16272:x1876702067561344:12345-192.168.203.37@tcp:101:dd.0 Request processed in 109us (189us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762970.377310:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 210 00000100:00100000:1.0:1789762970.380369:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.380404:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067561472 00000100:00100000:3.0:1789762970.380408:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067561472 00000100:00100000:3.0:1789762970.380431:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 211 00000100:00100000:3.0:1789762970.380436:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:16270:x1876702067561472:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762970.380446:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762970.380453:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000314ed567/0xdf5a7ac59f177a88 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb50 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.380467:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789762970.380470:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(2 0) 00010000:00010000:3.0:1789762970.380473:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000314ed567/0xdf5a7ac59f177a88 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb50 expref: 18 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.380481:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000314ed567/0xdf5a7ac59f177a88 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb50 expref: 18 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.380487:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8cae400, rc 0) 00000100:00100000:3.0:1789762970.380513:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:16270:x1876702067561472:12345-192.168.203.37@tcp:101:dd.0 Request processed in 75us (143us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762970.380520:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 211 00000100:00100000:0.0:1789762970.387296:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762970.387349:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067561600 00000100:00100000:2.0:1789762970.387355:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067561600 00000100:00100000:2.0:1789762970.387382:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 212 00000100:00100000:2.0:1789762970.387389:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:16271:x1876702067561600:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762970.387396:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762970.387404:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000651c861d/0xdf5a7ac59f177a8f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb57 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.387418:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(2 0) 00010000:00010000:2.0:1789762970.387421:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762970.387425:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000651c861d/0xdf5a7ac59f177a8f lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb57 expref: 19 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.387432:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000651c861d/0xdf5a7ac59f177a8f lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb57 expref: 19 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.387439:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c0031a00, rc 0) 00000100:00100000:2.0:1789762970.387465:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:16271:x1876702067561600:12345-192.168.203.37@tcp:101:dd.0 Request processed in 77us (171us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762970.387471:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 212 00000100:00100000:2.0:1789762970.408236:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762970.408281:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067561728 00000100:00100000:1.0:1789762970.408287:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067561728 00000100:00100000:1.0:1789762970.408318:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 213 00000100:00100000:1.0:1789762970.408326:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0ead180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:16278:x1876702067561728:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762970.408334:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762970.408342:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c6d86e53/0xdf5a7ac59f177a96 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb5e expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.408357:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(3 0) 00010000:00010000:1.0:1789762970.408360:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(4 0) 00010000:00010000:1.0:1789762970.408364:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c6d86e53/0xdf5a7ac59f177a96 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb5e expref: 20 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.408372:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c6d86e53/0xdf5a7ac59f177a96 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb5e expref: 20 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.408380:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee81b600, rc 0) 00000100:00100000:1.0:1789762970.408411:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0ead180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:16278:x1876702067561728:12345-192.168.203.37@tcp:101:dd.0 Request processed in 85us (177us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762970.408419:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 213 00000100:00100000:1.0:1789762970.416895:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.416946:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067561856 00000100:00100000:3.0:1789762970.416953:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067561856 00000100:00100000:3.0:1789762970.416995:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 214 00000100:00100000:3.0:1789762970.417010:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:16277:x1876702067561856:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:3.0:1789762970.417020:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762970.417030:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002040ccb8/0xdf5a7ac59f177a9d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb65 expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.417049:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(4 0) 00010000:00010000:3.0:1789762970.417054:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(5 0) 00010000:00010000:3.0:1789762970.417061:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002040ccb8/0xdf5a7ac59f177a9d lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 7 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb65 expref: 21 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.417073:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002040ccb8/0xdf5a7ac59f177a9d lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 7 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb65 expref: 21 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.417084:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8cae600, rc 0) 00000100:00100000:3.0:1789762970.417122:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:16277:x1876702067561856:12345-192.168.203.37@tcp:101:dd.0 Request processed in 118us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762970.417130:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 214 00000100:00100000:0.0:1789762970.419711:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762970.419800:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067561984 00000100:00100000:2.0:1789762970.419805:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067561984 00000100:00100000:2.0:1789762970.419836:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 215 00000100:00100000:2.0:1789762970.419842:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:16276:x1876702067561984:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:2.0:1789762970.419850:0:9038:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762970.419858:0:9038:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000024f15b62/0xdf5a7ac59f177aa4 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb6c expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.419871:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(5 0) 00010000:00010000:2.0:1789762970.419875:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:2.0:1789762970.419879:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000024f15b62/0xdf5a7ac59f177aa4 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 8 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb6c expref: 22 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.419887:0:9038:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000024f15b62/0xdf5a7ac59f177aa4 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 8 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb6c expref: 22 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.419893:0:9038:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c0031c00, rc 0) 00000100:00100000:2.0:1789762970.419941:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:16276:x1876702067561984:12345-192.168.203.37@tcp:101:dd.0 Request processed in 98us (230us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762970.419947:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 215 00000100:00100000:2.0:1789762970.428671:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762970.428718:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067562112 00000100:00100000:1.0:1789762970.428724:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067562112 00000100:00100000:1.0:1789762970.428752:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 216 00000100:00100000:1.0:1789762970.428759:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eaed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:16279:x1876702067562112:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762970.428768:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762970.428776:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ea99889e/0xdf5a7ac59f177aab lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb73 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.428792:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762970.428795:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762970.428799:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ea99889e/0xdf5a7ac59f177aab lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb73 expref: 23 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.428807:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ea99889e/0xdf5a7ac59f177aab lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb73 expref: 23 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.428814:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee81ae00, rc 0) 00000100:00100000:1.0:1789762970.428844:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eaed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:16279:x1876702067562112:12345-192.168.203.37@tcp:101:dd.0 Request processed in 84us (174us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762970.428851:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 216 00000100:00100000:1.0:1789762970.440750:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.440794:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067562240 00000100:00100000:3.0:1789762970.440800:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067562240 00002000:00100000:3.0:1789762970.440832:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0xa:0x0] (0->1048575) req@ffff9c37d184d500 x1876702067562240/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762970.440840:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184d500 with x1876702067562240 ext(0->1048575) 00002000:00010000:3.0:1789762970.440844:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184d500 00000100:00100000:3.0:1789762970.440858:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411596313039 time 411596389560 state 0x0 ref 1 00000100:00100000:3.0:1789762970.440864:0:8059:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 9: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763f00 gen 0 rate 20 depth 3 token 0 deadline 411596313039 time 411596389560 state 0x0 ref 1 00000100:00100000:3.0:1789762970.440887:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 3 deadline 443688060254 time 443687960254 state 0x1 ref 1 00000100:00100000:0.0:1789762970.441036:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443688060254 time 443688109470 state 0x0 ref 1 00000100:00100000:0.0:1789762970.441051:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:2396:x1876702067562240:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762970.445626:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:2396:x1876702067562240:12345-192.168.203.37@tcp:4:dd.0 Request processed in 4574us (4877us total) trans 4294968137 rc 0/0 00000100:00100000:2.0:1789762970.445637:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1140 00000100:00100000:2.0:1789762970.445645:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443688060254 time 443688109470 state 0x2 ref 0 00002000:00100000:2.0:1789762970.445655:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0xa:0x0] (0->1048575) req@ffff9c37d184d500 x1876702067562240/t4294968137(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.445662:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184d500 with x1876702067562240 ext(0->1048575) 00002000:00010000:2.0:1789762970.445667:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184d500 00000100:00100000:0.0:1789762970.451102:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.451142:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067562368 00000100:00100000:3.0:1789762970.451148:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067562368 00002000:00100000:3.0:1789762970.451172:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0xa:0x0] (1048576->2097151) req@ffff9c36d3d68380 x1876702067562368/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762970.451185:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d68380 with x1876702067562368 ext(1048576->2097151) 00002000:00010000:3.0:1789762970.451188:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36d3d68380 00000100:00100000:3.0:1789762970.451197:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443688060254 time 443688109470 state 0x0 ref 1 00000100:00100000:3.0:1789762970.451223:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443688209470 time 443688109470 state 0x1 ref 1 00000100:00100000:3.0:1789762970.451240:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443688209470 time 443698318645 state 0x0 ref 1 00000100:00100000:3.0:1789762970.451250:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:2395:x1876702067562368:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762970.453561:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:2395:x1876702067562368:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2311us (2459us total) trans 4294968138 rc 0/0 00000100:00100000:3.0:1789762970.453567:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1141 00000100:00100000:3.0:1789762970.453573:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443688209470 time 443698318645 state 0x2 ref 0 00002000:00100000:3.0:1789762970.453595:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0xa:0x0] (1048576->2097151) req@ffff9c36d3d68380 x1876702067562368/t4294968138(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762970.453603:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d68380 with x1876702067562368 ext(1048576->2097151) 00002000:00010000:3.0:1789762970.453606:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36d3d68380 00000100:00100000:0.0:1789762970.453618:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762970.453671:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067562496 00000100:00100000:2.0:1789762970.453677:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067562496 00002000:00100000:2.0:1789762970.453700:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0xa:0x0] (4194304->5242879) req@ffff9c36d3d6a680 x1876702067562496/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.453710:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d6a680 with x1876702067562496 ext(4194304->5242879) 00002000:00010000:2.0:1789762970.453715:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36d3d6a680 00000100:00100000:2.0:1789762970.453739:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443688209470 time 443698318645 state 0x0 ref 1 00000100:00100000:2.0:1789762970.453774:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443698418645 time 443698318645 state 0x1 ref 1 00000100:00100000:3.0:1789762970.453790:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443698418645 time 443700865559 state 0x0 ref 1 00000100:00100000:3.0:1789762970.453802:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:2397:x1876702067562496:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762970.456182:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2397:x1876702067562496:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2381us (2567us total) trans 4294968139 rc 0/0 00000100:00100000:3.0:1789762970.456189:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1142 00000100:00100000:3.0:1789762970.456195:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443698418645 time 443700865559 state 0x2 ref 0 00002000:00100000:3.0:1789762970.456203:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0xa:0x0] (4194304->5242879) req@ffff9c36d3d6a680 x1876702067562496/t4294968139(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762970.456211:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d6a680 with x1876702067562496 ext(4194304->5242879) 00002000:00010000:3.0:1789762970.456214:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36d3d6a680 00000100:00100000:1.0:1789762970.462335:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762970.462385:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067562624 00000100:00100000:2.0:1789762970.462394:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067562624 00002000:00100000:2.0:1789762970.462432:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0xa:0x0] (2097152->3145727) req@ffff9c37d184dc00 x1876702067562624/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.462440:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184dc00 with x1876702067562624 ext(2097152->3145727) 00002000:00010000:2.0:1789762970.462444:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184dc00 00000100:00100000:2.0:1789762970.462457:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443698418645 time 443700865559 state 0x0 ref 1 00000100:00100000:2.0:1789762970.462492:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443700965559 time 443700865559 state 0x1 ref 1 00000100:00100000:1.0:1789762970.462524:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443700965559 time 443709597738 state 0x0 ref 1 00000100:00100000:1.0:1789762970.462538:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2396:x1876702067562624:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:0.0:1789762970.464450:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.464774:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067562752 00000100:00100000:3.0:1789762970.464780:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067562752 00002000:00100000:3.0:1789762970.464809:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0xa:0x0] (7340032->8388607) req@ffff9c36d3d6bb80 x1876702067562752/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762970.464820:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d6bb80 with x1876702067562752 ext(7340032->8388607) 00002000:00010000:3.0:1789762970.464826:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36d3d6bb80 00000100:00100000:3.0:1789762970.464870:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443709697738 time 443709597738 state 0x1 ref 2 00000100:00100000:3.0:1789762970.464904:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443709697738 time 443711980179 state 0x0 ref 2 00000100:00100000:3.0:1789762970.464989:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2398:x1876702067562752:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762970.465433:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:2396:x1876702067562624:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2896us (3102us total) trans 4294968140 rc 0/0 00000100:00100000:2.0:1789762970.465441:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1143 00002000:00100000:2.0:1789762970.465448:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0xa:0x0] (2097152->3145727) req@ffff9c37d184dc00 x1876702067562624/t4294968140(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.465455:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184dc00 with x1876702067562624 ext(2097152->3145727) 00002000:00010000:2.0:1789762970.465459:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184dc00 00000100:00100000:2.0:1789762970.466258:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.466295:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067562880 00000100:00100000:3.0:1789762970.466302:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067562880 00002000:00100000:3.0:1789762970.467068:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0xa:0x0] (6291456->7340031) req@ffff9c37e0eaea00 x1876702067562880/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762970.467088:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e0eaea00 with x1876702067562880 ext(6291456->7340031) 00002000:00010000:3.0:1789762970.467093:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e0eaea00 00000100:00100000:3.0:1789762970.467126:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443712080179 time 443711980179 state 0x1 ref 2 00000100:00100000:2.0:1789762970.467160:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443712080179 time 443714234965 state 0x0 ref 2 00000100:00100000:2.0:1789762970.467175:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eaea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:2395:x1876702067562880:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762970.469356:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2398:x1876702067562752:12345-192.168.203.37@tcp:4:dd.0 Request processed in 4437us (4906us total) trans 4294968141 rc 0/0 00000100:00100000:3.0:1789762970.469368:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1144 00002000:00100000:3.0:1789762970.469379:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0xa:0x0] (7340032->8388607) req@ffff9c36d3d6bb80 x1876702067562752/t4294968141(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762970.469390:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d6bb80 with x1876702067562752 ext(7340032->8388607) 00002000:00010000:3.0:1789762970.469397:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36d3d6bb80 00000100:00100000:2.0:1789762970.469540:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eaea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2395:x1876702067562880:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2369us (3282us total) trans 4294968142 rc 0/0 00000100:00100000:2.0:1789762970.469547:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1145 00000100:00100000:2.0:1789762970.469553:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443712080179 time 443714234965 state 0x2 ref 0 00002000:00100000:2.0:1789762970.469562:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0xa:0x0] (6291456->7340031) req@ffff9c37e0eaea00 x1876702067562880/t4294968142(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.469569:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37e0eaea00 with x1876702067562880 ext(6291456->7340031) 00002000:00010000:2.0:1789762970.469572:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37e0eaea00 00000100:00100000:0.0:1789762970.474170:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762970.474218:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067563008 00000100:00100000:1.0:1789762970.474223:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067563008 00002000:00100000:1.0:1789762970.474255:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0xa:0x0] (5242880->6291455) req@ffff9c36d3d69880 x1876702067563008/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762970.474266:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d69880 with x1876702067563008 ext(5242880->6291455) 00002000:00010000:1.0:1789762970.474271:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36d3d69880 00000100:00100000:1.0:1789762970.474283:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443712080179 time 443714234965 state 0x0 ref 1 00000100:00100000:1.0:1789762970.474317:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443714334965 time 443714234965 state 0x1 ref 1 00000100:00100000:1.0:1789762970.474340:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443714334965 time 443721418511 state 0x0 ref 1 00000100:00100000:1.0:1789762970.474352:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2396:x1876702067563008:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:1.0:1789762970.475632:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.475670:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067563136 00000100:00100000:3.0:1789762970.475676:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067563136 00002000:00100000:3.0:1789762970.475700:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0xa:0x0] (3145728->4194303) req@ffff9c37d184f480 x1876702067563136/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762970.475711:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184f480 with x1876702067563136 ext(3145728->4194303) 00002000:00010000:3.0:1789762970.475715:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184f480 00000100:00100000:3.0:1789762970.475748:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443721518511 time 443721418511 state 0x1 ref 2 00000100:00100000:3.0:1789762970.475767:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443721518511 time 443722845332 state 0x0 ref 2 00000100:00100000:3.0:1789762970.475778:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+32:2397:x1876702067563136:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762970.480335:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2397:x1876702067563136:12345-192.168.203.37@tcp:4:dd.0 Request processed in 4554us (4701us total) trans 4294968143 rc 0/0 00000100:00100000:2.0:1789762970.480343:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1147 00002000:00100000:2.0:1789762970.480352:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0xa:0x0] (3145728->4194303) req@ffff9c37d184f480 x1876702067563136/t4294968143(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.480360:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184f480 with x1876702067563136 ext(3145728->4194303) 00002000:00010000:2.0:1789762970.480364:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37d184f480 00000100:00100000:0.0:1789762970.480696:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2396:x1876702067563008:12345-192.168.203.37@tcp:4:dd.0 Request processed in 6345us (6526us total) trans 4294968144 rc 0/0 00000100:00100000:0.0:1789762970.480706:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1146 00000100:00100000:0.0:1789762970.480716:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443721518511 time 443722845332 state 0x2 ref 0 00002000:00100000:0.0:1789762970.480728:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0xa:0x0] (5242880->6291455) req@ffff9c36d3d69880 x1876702067563008/t4294968144(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762970.480738:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d69880 with x1876702067563008 ext(5242880->6291455) 00002000:00010000:0.0:1789762970.480743:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36d3d69880 00000100:00100000:2.0:1789762970.500352:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 443947408137 time 443747408137 state 0x1 ref 9 00000100:00100000:2.0:1789762970.500375:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36c11d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+22:2397:x1876702067559424:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762970.503645:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36c11d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:2397:x1876702067559424:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3266us (566375us total) trans 4294967421 rc 0/0 00000100:00100000:2.0:1789762970.503653:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1131 00002000:00100000:2.0:1789762970.503663:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x2:0x0] (3145728->4194303) req@ffff9c36c11d8700 x1876702067559424/t4294967421(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/448 e 0 to 0 dl 1789762980 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762970.503671:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36c11d8700 with x1876702067559424 ext(3145728->4194303) 00002000:00010000:2.0:1789762970.503677:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36c11d8700 00000100:00100000:0.0:1789762970.506525:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762970.506613:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067563264 00000100:00100000:2.0:1789762970.506634:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067563264 00002000:00100000:2.0:1789762970.506662:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x3:0x0] (0->1048575) req@ffff9c36d3d6b800 x1876702067563264/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.506670:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d6b800 with x1876702067563264 ext(0->1048575) 00010000:00010000:2.0:1789762970.506677:0:13300:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 24 pid: 8053 timeout: 543 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.506695:0:13300:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 24 pid: 8053 timeout: 543 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762970.506701:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9c36d3d6b800 00000100:00100000:2.0:1789762970.506713:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763200 gen 0 rate 20 depth 3 token 1 deadline 365751187388 time 365736974958 state 0x0 ref 1 00000100:00100000:2.0:1789762970.506719:0:13300:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 9: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763200 gen 0 rate 20 depth 3 token 1 deadline 365751187388 time 365736974958 state 0x0 ref 1 00000100:00100000:2.0:1789762970.506737:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 3 deadline 443753913781 time 443753813781 state 0x1 ref 1 00000100:00100000:2.0:1789762970.506769:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 443753913781 time 443753847629 state 0x0 ref 1 00000100:00100000:2.0:1789762970.506778:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:2397:x1876702067563264:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762970.511095:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2397:x1876702067563264:12345-192.168.203.37@tcp:4:dd.0 Request processed in 4315us (4569us total) trans 4294967422 rc 0/0 00000100:00100000:2.0:1789762970.511103:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 8 00000100:00100000:2.0:1789762970.511110:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c36ca126600 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34c00 gen 0 rate 10000 depth 3 token 2 deadline 443753913781 time 443753847629 state 0x2 ref 0 00002000:00100000:2.0:1789762970.511119:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x3:0x0] (0->1048575) req@ffff9c36d3d6b800 x1876702067563264/t4294967422(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.511126:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d6b800 with x1876702067563264 ext(0->1048575) 00010000:00010000:2.0:1789762970.511133:0:13300:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 26 pid: 8053 timeout: 543 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.511148:0:13300:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 26 pid: 8053 timeout: 543 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762970.511154:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9c36d3d6b800 00000100:00100000:1.0:1789762970.512598:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762970.512657:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067563392 00000100:00100000:2.0:1789762970.512663:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067563392 00010000:00010000:2.0:1789762970.512695:0:6404:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 26 pid: 8053 timeout: 543 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762970.512721:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 91 00000100:00100000:2.0:1789762970.512727:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+26:2396:x1876702067563392:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762970.512736:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1789762970.512746:0:6404:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000401:0x3:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:2.0:1789762970.512749:0:6404:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000401:0x3:0x0] updating lvb mtime from disk: 0 -> 1789762970 00002000:00010000:2.0:1789762970.512751:0:6404:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000401:0x3:0x0] updating lvb ctime from disk: 0 -> 1789762970 00010000:00010000:2.0:1789762970.512755:0:6404:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 26 pid: 8053 timeout: 543 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512766:0:6404:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 26 pid: 8053 timeout: 543 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512796:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bec972d4/0xdf5a7ac59f177a73 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb3b expref: 26 pid: 8053 timeout: 543 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512810:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512815:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512821:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000cc930a86/0xdf5a7ac59f177a7a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb42 expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512829:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cc930a86/0xdf5a7ac59f177a7a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb42 expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512836:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512837:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512841:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000314ed567/0xdf5a7ac59f177a88 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb50 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512847:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000314ed567/0xdf5a7ac59f177a88 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb50 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512851:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512852:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512855:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000651c861d/0xdf5a7ac59f177a8f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb57 expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512861:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000651c861d/0xdf5a7ac59f177a8f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb57 expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512865:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512866:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512869:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c6d86e53/0xdf5a7ac59f177a96 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb5e expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512879:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c6d86e53/0xdf5a7ac59f177a96 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb5e expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512883:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512885:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512888:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002040ccb8/0xdf5a7ac59f177a9d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb65 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512893:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002040ccb8/0xdf5a7ac59f177a9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb65 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512897:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512898:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512901:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000024f15b62/0xdf5a7ac59f177aa4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb6c expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512906:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000024f15b62/0xdf5a7ac59f177aa4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb6c expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512910:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512912:0:6404:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762970.512915:0:6404:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ea99889e/0xdf5a7ac59f177aab lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb73 expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512958:0:6404:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ea99889e/0xdf5a7ac59f177aab lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb73 expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.512985:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ea99889e/0xdf5a7ac59f177aab lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb73 expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762970.512996:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0ead500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074760704:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762970.513075:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000024f15b62/0xdf5a7ac59f177aa4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb6c expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762970.513089:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eafb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074760832:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762970.513124:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002040ccb8/0xdf5a7ac59f177a9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb65 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762970.513132:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eac700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074760960:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762970.513158:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c6d86e53/0xdf5a7ac59f177a96 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb5e expref: 25 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762970.513165:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eaf800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074761088:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762970.513207:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000651c861d/0xdf5a7ac59f177a8f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb57 expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762970.513218:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eadf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074761216:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762970.513258:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000314ed567/0xdf5a7ac59f177a88 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb50 expref: 25 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762970.513276:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eae300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074761344:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762970.513307:0:6404:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000cc930a86/0xdf5a7ac59f177a7a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb42 expref: 25 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762970.513318:0:6404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eae680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074761472:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762970.513340:0:6404:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d2724a80 going to sleep for 11 seconds 00000100:00100000:2.0:1789762970.515829:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0ead500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074760704:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762970.515842:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eafb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074760832:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762970.515848:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eac700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074760960:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762970.515853:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eaf800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074761088:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762970.515858:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eadf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074761216:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762970.515862:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eae300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074761344:192.168.203.37@tcp:105: 00000100:00100000:2.0:1789762970.515866:0:6404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eae680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6404:1876702074761472:192.168.203.37@tcp:105: 00010000:00010000:2.0:1789762970.515887:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762970.515914:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2396:x1876702067563392:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 3188us (3318us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762970.515940:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 91 00000100:00100000:1.0:1789762970.561061:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.561112:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067563520 00000100:00100000:3.0:1789762970.561122:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067563520 00002000:00100000:3.0:1789762970.561172:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x3:0x0] (3145728->4194303) req@ffff9c37c962b480 x1876702067563520/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762970.561186:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c962b480 with x1876702067563520 ext(3145728->4194303) 00002000:00010000:3.0:1789762970.561191:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c962b480 00000100:00100000:3.0:1789762970.561207:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443721518511 time 443722845332 state 0x0 ref 1 00000100:00100000:3.0:1789762970.561249:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443722945332 time 443722845332 state 0x1 ref 1 00000100:00100000:3.0:1789762970.561279:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443722945332 time 443808355970 state 0x0 ref 1 00000100:00100000:3.0:1789762970.561291:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2396:x1876702067563520:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762970.564651:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2396:x1876702067563520:12345-192.168.203.37@tcp:4:dd.0 Request processed in 3361us (3597us total) trans 4294967423 rc 0/0 00000100:00100000:3.0:1789762970.564658:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1148 00000100:00100000:3.0:1789762970.564664:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443722945332 time 443808355970 state 0x2 ref 0 00002000:00100000:3.0:1789762970.564673:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x3:0x0] (3145728->4194303) req@ffff9c37c962b480 x1876702067563520/t4294967423(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762970.564680:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c962b480 with x1876702067563520 ext(3145728->4194303) 00002000:00010000:3.0:1789762970.564683:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c962b480 00000100:00100000:0.0:1789762970.578861:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762970.578905:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067563648 00000100:00100000:1.0:1789762970.578911:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067563648 00002000:00100000:1.0:1789762970.578948:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x3:0x0] (2097152->3145727) req@ffff9c36d3d69f80 x1876702067563648/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762970.578965:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d69f80 with x1876702067563648 ext(2097152->3145727) 00002000:00010000:1.0:1789762970.578969:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36d3d69f80 00000100:00100000:1.0:1789762970.578982:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443722945332 time 443808355970 state 0x0 ref 1 00000100:00100000:1.0:1789762970.579015:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443808455970 time 443808355970 state 0x1 ref 1 00000100:00100000:1.0:1789762970.579040:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443808455970 time 443826118583 state 0x0 ref 1 00000100:00100000:1.0:1789762970.579052:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+27:2395:x1876702067563648:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762970.581431:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+29:2395:x1876702067563648:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2380us (2570us total) trans 4294967424 rc 0/0 00000100:00100000:2.0:1789762970.581439:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1149 00000100:00100000:2.0:1789762970.581447:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443808455970 time 443826118583 state 0x2 ref 0 00002000:00100000:2.0:1789762970.581456:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x3:0x0] (2097152->3145727) req@ffff9c36d3d69f80 x1876702067563648/t4294967424(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.581463:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d69f80 with x1876702067563648 ext(2097152->3145727) 00002000:00010000:2.0:1789762970.581467:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36d3d69f80 00000100:00100000:1.0:1789762970.583857:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762970.583958:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067563776 00000100:00100000:2.0:1789762970.583964:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067563776 00000100:00100000:0.0:1789762970.583978:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00002000:00100000:2.0:1789762970.583987:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x3:0x0] (7340032->8388607) req@ffff9c37c9629f80 x1876702067563776/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.583996:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c9629f80 with x1876702067563776 ext(7340032->8388607) 00002000:00010000:2.0:1789762970.584000:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c9629f80 00000100:00100000:2.0:1789762970.584008:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443808455970 time 443826118583 state 0x0 ref 1 00000100:00100000:3.0:1789762970.584014:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067563904 00000100:00100000:3.0:1789762970.584021:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067563904 00000100:00100000:2.0:1789762970.584039:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443826218583 time 443826118583 state 0x1 ref 1 00002000:00100000:3.0:1789762970.584045:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x3:0x0] (5242880->6291455) req@ffff9c36d3d68700 x1876702067563904/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762970.584057:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d68700 with x1876702067563904 ext(5242880->6291455) 00002000:00010000:3.0:1789762970.584062:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36d3d68700 00000100:00100000:2.0:1789762970.584069:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443826218583 time 443831142280 state 0x0 ref 1 00000100:00100000:2.0:1789762970.584081:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9629f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+30:2397:x1876702067563776:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:3.0:1789762970.584096:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443831242280 time 443831142280 state 0x1 ref 2 00000100:00100000:3.0:1789762970.584200:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443831242280 time 443831273560 state 0x0 ref 2 00000100:00100000:3.0:1789762970.584213:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d3d68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+31:2397:x1876702067563904:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762970.586363:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9629f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:2397:x1876702067563776:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2281us (2502us total) trans 4294967425 rc 0/0 00000100:00100000:2.0:1789762970.586373:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1150 00002000:00100000:2.0:1789762970.586383:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x3:0x0] (7340032->8388607) req@ffff9c37c9629f80 x1876702067563776/t4294967425(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.586397:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c9629f80 with x1876702067563776 ext(7340032->8388607) 00002000:00010000:2.0:1789762970.586401:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c9629f80 00000100:00100000:1.0:1789762970.586886:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762970.586937:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067564032 00000100:00100000:2.0:1789762970.586942:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067564032 00002000:00100000:2.0:1789762970.586962:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x3:0x0] (1048576->2097151) req@ffff9c37c962b100 x1876702067564032/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.586969:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c962b100 with x1876702067564032 ext(1048576->2097151) 00002000:00010000:2.0:1789762970.586972:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c962b100 00000100:00100000:2.0:1789762970.586999:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443831373560 time 443831273560 state 0x1 ref 2 00000100:00100000:2.0:1789762970.587023:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443831373560 time 443834101286 state 0x0 ref 2 00000100:00100000:2.0:1789762970.587034:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+33:2396:x1876702067564032:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762970.588598:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d3d68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2397:x1876702067563904:12345-192.168.203.37@tcp:4:dd.0 Request processed in 4386us (4621us total) trans 4294967426 rc 0/0 00000100:00100000:2.0:1789762970.588606:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1151 00002000:00100000:2.0:1789762970.588613:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x3:0x0] (5242880->6291455) req@ffff9c36d3d68700 x1876702067563904/t4294967426(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.588620:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36d3d68700 with x1876702067563904 ext(5242880->6291455) 00002000:00010000:2.0:1789762970.588624:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36d3d68700 00000100:00100000:2.0:1789762970.589497:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2396:x1876702067564032:12345-192.168.203.37@tcp:4:dd.0 Request processed in 2464us (2610us total) trans 4294967427 rc 0/0 00000100:00100000:2.0:1789762970.589505:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1152 00000100:00100000:2.0:1789762970.589512:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443831373560 time 443834101286 state 0x2 ref 0 00002000:00100000:2.0:1789762970.589521:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x3:0x0] (1048576->2097151) req@ffff9c37c962b100 x1876702067564032/t4294967427(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.589528:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c962b100 with x1876702067564032 ext(1048576->2097151) 00002000:00010000:2.0:1789762970.589531:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c962b100 00000100:00100000:0.0:1789762970.593115:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762970.593162:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067564416 00000100:00100000:2.0:1789762970.593168:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067564416 00000100:00100000:2.0:1789762970.593201:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 109 00000100:00100000:2.0:1789762970.593209:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:16278:x1876702067564416:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:2.0:1789762970.593442:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:16278:x1876702067564416:12345-192.168.203.37@tcp:35:dd.0 Request processed in 234us (329us total) trans 4294967458 rc 0/0 00000100:00100000:2.0:1789762970.593448:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 109 00000100:00100000:1.0:1789762970.593952:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762970.593984:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762970.594006:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067564160 00000100:00100000:3.0:1789762970.594014:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067564160 00000100:00100000:0.0:1789762970.594032:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067564544 00000100:00100000:0.0:1789762970.594037:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067564544 00002000:00100000:3.0:1789762970.594039:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x3:0x0] (6291456->7340031) req@ffff9c37c962a300 x1876702067564160/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762970.594050:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c962a300 with x1876702067564160 ext(6291456->7340031) 00002000:00010000:3.0:1789762970.594054:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c962a300 00002000:00100000:0.0:1789762970.594055:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x3:0x0] (4194304->5242879) req@ffff9c37c9628380 x1876702067564544/t0(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/0 e 0 to 0 dl 1789762981 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762970.594064:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c9628380 with x1876702067564544 ext(4194304->5242879) 00000100:00100000:3.0:1789762970.594066:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443831373560 time 443834101286 state 0x0 ref 1 00002000:00010000:0.0:1789762970.594068:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c9628380 00000100:00100000:3.0:1789762970.594099:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443834201286 time 443834101286 state 0x1 ref 2 00000100:00100000:0.0:1789762970.594110:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443834201286 time 443834101286 state 0x1 ref 2 00000100:00100000:3.0:1789762970.594122:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 443841299384 time 443841199384 state 0x1 ref 2 00000100:00100000:3.0:1789762970.594135:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2397:x1876702067564160:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762970.594148:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 1 deadline 443841299384 time 443841217291 state 0x0 ref 2 00000100:00100000:2.0:1789762970.594165:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2397:x1876702067564544:12345-192.168.203.37@tcp:4:dd.0 00000100:00100000:2.0:1789762970.600434:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2397:x1876702067564160:12345-192.168.203.37@tcp:4:dd.0 Request processed in 6299us (6482us total) trans 4294967428 rc 0/0 00000100:00100000:2.0:1789762970.600443:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1153 00002000:00100000:2.0:1789762970.600450:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x3:0x0] (6291456->7340031) req@ffff9c37c962a300 x1876702067564160/t4294967428(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.600458:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c962a300 with x1876702067564160 ext(6291456->7340031) 00002000:00010000:2.0:1789762970.600462:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c962a300 00000100:00100000:2.0:1789762970.601246:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2397:x1876702067564544:12345-192.168.203.37@tcp:4:dd.0 Request processed in 7085us (7260us total) trans 4294967429 rc 0/0 00000100:00100000:2.0:1789762970.601255:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1154 00000100:00100000:2.0:1789762970.601265:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff76f400 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 1 deadline 443841299384 time 443841217291 state 0x2 ref 0 00002000:00100000:2.0:1789762970.601277:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x3:0x0] (4194304->5242879) req@ffff9c37c9628380 x1876702067564544/t4294967429(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762970.601287:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c9628380 with x1876702067564544 ext(4194304->5242879) 00002000:00010000:2.0:1789762970.601292:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37c9628380 00000100:00100000:2.0:1789762970.700425:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 444147487023 time 443947487023 state 0x1 ref 8 00000100:00100000:2.0:1789762970.700451:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb450700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+34:2398:x1876702067559552:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762970.703516:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb450700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2398:x1876702067559552:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3075us (759365us total) trans 4294968145 rc 0/0 00000100:00100000:2.0:1789762970.703524:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1132 00002000:00100000:2.0:1789762970.703533:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x9:0x0] (4194304->5242879) req@ffff9c36cb450700 x1876702067559552/t4294968145(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/448 e 0 to 0 dl 1789762980 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762970.703541:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb450700 with x1876702067559552 ext(4194304->5242879) 00002000:00010000:2.0:1789762970.703546:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36cb450700 00000100:00100000:2.0:1789762970.900486:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 444347550497 time 444147550497 state 0x1 ref 7 00000100:00100000:2.0:1789762970.900512:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb592d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:2397:x1876702067559680:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762970.903172:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb592d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2397:x1876702067559680:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2669us (949116us total) trans 4294968146 rc 0/0 00000100:00100000:2.0:1789762970.903180:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1133 00002000:00100000:2.0:1789762970.903190:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x9:0x0] (5242880->6291455) req@ffff9c36cb592d80 x1876702067559680/t4294968146(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/448 e 0 to 0 dl 1789762980 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762970.903198:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb592d80 with x1876702067559680 ext(5242880->6291455) 00002000:00010000:2.0:1789762970.903204:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36cb592d80 00000100:00100000:1.0:1789762971.080015:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762971.080050:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762971.080071:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762971.080099:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762971.080101:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067564800 00000100:00100000:2.0:1789762971.080125:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067564672 00000100:00100000:3.0:1789762971.080130:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067564800 00000100:00100000:2.0:1789762971.080139:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067564672 00000100:00100000:0.0:1789762971.080144:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067565056 00000100:00100000:0.0:1789762971.080156:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067565056 00000100:00100000:1.0:1789762971.080215:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067564928 00000100:00100000:1.0:1789762971.080230:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067564928 00000100:00100000:2.0:1789762971.080245:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 399 00000100:00100000:3.0:1789762971.080287:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 688 00000100:00100000:2.0:1789762971.080290:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067564672:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762971.080299:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 689 00000100:00100000:3.0:1789762971.080301:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9629880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067564800:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762971.080313:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9628000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067565056:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762971.080324:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 690 00000100:00100000:1.0:1789762971.080334:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9628a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067564928:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762971.080363:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9629880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2396:x1876702067564800:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 66us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762971.080368:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9628000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067565056:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 55us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762971.080370:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2395:x1876702067564672:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 106us (356us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762971.080376:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9628a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067564928:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 41us (274us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762971.080381:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 688 00000100:00100000:0.0:1789762971.080421:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 689 00000100:00100000:2.0:1789762971.080424:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 399 00000100:00100000:1.0:1789762971.080437:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 690 00000100:00080000:1.0:1789762971.080492:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762971.080499:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762971.100552:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 444547619293 time 444347619293 state 0x1 ref 6 00000100:00100000:0.0:1789762971.100589:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+36:2398:x1876702067559808:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:1.0:1789762971.100630:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762971.103313:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2398:x1876702067559808:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2747us (1142850us total) trans 4294968147 rc 0/0 00000100:00100000:0.0:1789762971.103324:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1134 00002000:00100000:0.0:1789762971.103335:0:14171:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x9:0x0] (6291456->7340031) req@ffff9c37f7f5ad80 x1876702067559808/t4294968147(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/448 e 0 to 0 dl 1789762980 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762971.103347:0:14171:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f7f5ad80 with x1876702067559808 ext(6291456->7340031) 00002000:00010000:0.0:1789762971.103354:0:14171:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37f7f5ad80 00010000:00010000:0.0:1789762971.264074:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762971.300729:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 444747786350 time 444547786350 state 0x1 ref 5 00000100:00100000:1.0:1789762971.300771:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:2397:x1876702067559936:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762971.305816:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2397:x1876702067559936:12345-192.168.203.37@tcp:4:dd.500 Request processed in 5058us (1338619us total) trans 4294968148 rc 0/0 00000100:00100000:1.0:1789762971.305831:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1135 00002000:00100000:1.0:1789762971.305847:0:14170:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x9:0x0] (7340032->8388607) req@ffff9c36cb453800 x1876702067559936/t4294968148(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:750/0 lens 488/448 e 0 to 0 dl 1789762980 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762971.305863:0:14170:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb453800 with x1876702067559936 ext(7340032->8388607) 00002000:00010000:1.0:1789762971.305872:0:14170:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c36cb453800 00000100:00100000:1.0:1789762971.500867:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 444947928045 time 444747928045 state 0x1 ref 4 00000100:00080000:2.0:1789762971.500947:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762971.500961:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7f5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:2398:x1876702067560064:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:1.0:1789762971.504135:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7f5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2398:x1876702067560064:12345-192.168.203.37@tcp:4:dd.500 Request processed in 3250us (1397997us total) trans 4294967430 rc 0/0 00000100:00100000:1.0:1789762971.504144:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1136 00002000:00100000:1.0:1789762971.504155:0:8059:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x2:0x0] (4194304->5242879) req@ffff9c37f7f5b800 x1876702067560064/t4294967430(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762971.504164:0:8059:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37f7f5b800 with x1876702067560064 ext(4194304->5242879) 00002000:00010000:1.0:1789762971.504170:0:8059:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37f7f5b800 00000100:00100000:2.0:1789762971.700982:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 445148045932 time 444948045932 state 0x1 ref 3 00000100:00100000:2.0:1789762971.701009:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2398:x1876702067560192:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:2.0:1789762971.703766:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2398:x1876702067560192:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2767us (1589092us total) trans 4294967431 rc 0/0 00000100:00100000:2.0:1789762971.703775:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1137 00002000:00100000:2.0:1789762971.703785:0:8058:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x2:0x0] (5242880->6291455) req@ffff9c37d184c700 x1876702067560192/t4294967431(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762971.703793:0:8058:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184c700 with x1876702067560192 ext(5242880->6291455) 00002000:00010000:2.0:1789762971.703799:0:8058:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184c700 00000100:00100000:3.0:1789762971.901651:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 445348709599 time 445148709599 state 0x1 ref 2 00000100:00100000:3.0:1789762971.901680:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+40:2397:x1876702067560320:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762971.904590:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+42:2397:x1876702067560320:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2921us (1597852us total) trans 4294967432 rc 0/0 00000100:00100000:3.0:1789762971.904599:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1138 00002000:00100000:3.0:1789762971.904608:0:13300:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x2:0x0] (6291456->7340031) req@ffff9c37d184d180 x1876702067560320/t4294967432(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762971.904617:0:13300:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37d184d180 with x1876702067560320 ext(6291456->7340031) 00002000:00010000:3.0:1789762971.904622:0:13300:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c37d184d180 00000100:00100000:2.0:1789762972.101707:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 445348709599 time 445348773394 state 0x0 ref 1 00000100:00100000:2.0:1789762972.101734:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb452300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+41:2397:x1876702067560448:12345-192.168.203.37@tcp:4:dd.500 00000100:00100000:3.0:1789762972.104474:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb452300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:2397:x1876702067560448:12345-192.168.203.37@tcp:4:dd.500 Request processed in 2751us (1790330us total) trans 4294967433 rc 0/0 00000100:00100000:3.0:1789762972.104484:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1139 00000100:00100000:3.0:1789762972.104492:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 445348709599 time 445348773394 state 0x2 ref 0 00002000:00100000:3.0:1789762972.104503:0:13301:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x2:0x0] (7340032->8388607) req@ffff9c36cb452300 x1876702067560448/t4294967433(0) o4->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:751/0 lens 488/448 e 0 to 0 dl 1789762981 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762972.104514:0:13301:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c36cb452300 with x1876702067560448 ext(7340032->8388607) 00002000:00010000:3.0:1789762972.104522:0:13301:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9c36cb452300 00000100:00100000:1.0:1789762972.154362:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762972.154413:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067565184 00000100:00100000:3.0:1789762972.154421:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067565184 00000100:00100000:3.0:1789762972.154473:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 77 00000100:00100000:3.0:1789762972.154481:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9629180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+43:6823:x1876702067565184:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762972.154493:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 8 locks, starting at 0 00002000:00010000:3.0:1789762972.154511:0:7218:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000401:0x2:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:3.0:1789762972.154515:0:7218:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000401:0x2:0x0] updating lvb mtime from disk: 0 -> 1789762970 00002000:00010000:3.0:1789762972.154518:0:7218:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000401:0x2:0x0] updating lvb ctime from disk: 0 -> 1789762970 00002000:00010000:3.0:1789762972.154520:0:7218:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000401:0x2:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:3.0:1789762972.154547:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a558e1d5/0xdf5a7ac59f177a49 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb2d expref: 43 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762972.154593:0:7218:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x2c0000401:0x3:0x0] updating lvb size from disk: 1048576 -> 8388608 00010000:00010000:3.0:1789762972.154606:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024f15b62/0xdf5a7ac59f177aa4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb6c expref: 42 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.154621:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c6d86e53/0xdf5a7ac59f177a96 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb5e expref: 41 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.154635:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cc930a86/0xdf5a7ac59f177a7a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 7 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb42 expref: 40 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.154644:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002040ccb8/0xdf5a7ac59f177a9d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 6 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb65 expref: 39 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.154657:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ea99889e/0xdf5a7ac59f177aab lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 5 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb73 expref: 38 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.154665:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000314ed567/0xdf5a7ac59f177a88 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb50 expref: 37 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.154672:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000651c861d/0xdf5a7ac59f177a8f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb57 expref: 36 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.154680:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762972.154722:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9629180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+35:6823:x1876702067565184:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 242us (368us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762972.154727:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 77 00000100:00100000:2.0:1789762972.154854:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.154905:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067565312 00000100:00100000:1.0:1789762972.154910:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067565312 00000100:00100000:1.0:1789762972.154984:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 78 00000100:00100000:1.0:1789762972.154994:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eae680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:6358:x1876702067565312:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762972.155004:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00002000:00010000:1.0:1789762972.155016:0:6405:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x9:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:1.0:1789762972.155019:0:6405:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x9:0x0] updating lvb mtime from disk: 0 -> 1789762969 00002000:00010000:1.0:1789762972.155021:0:6405:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x9:0x0] updating lvb ctime from disk: 0 -> 1789762969 00010000:00010000:1.0:1789762972.155037:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177a3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb26 expref: 39 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762972.155059:0:6405:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0xa:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:1.0:1789762972.155062:0:6405:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0xa:0x0] updating lvb mtime from disk: 0 -> 1789762970 00002000:00010000:1.0:1789762972.155064:0:6405:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0xa:0x0] updating lvb ctime from disk: 0 -> 1789762970 00010000:00010000:1.0:1789762972.155073:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177a81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb49 expref: 38 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.155085:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762972.155118:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eae680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:6358:x1876702067565312:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 126us (267us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.155126:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 78 00000100:00100000:1.0:1789762972.160044:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36ca1afb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074761856:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762972.160049:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074761728:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762972.160082:0:3427:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789762972.160096:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eae300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074761984:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789762972.160114:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762972.160145:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762972.160196:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eac700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074762368:0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.160205:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eadf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074762112:0@lo:400:kworker.0 00000100:00100000:2.0:1789762972.160256:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eafb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074762496:0@lo:400:kworker.0 00000100:00100000:1.0:1789762972.160259:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762972.160261:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762972.160292:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762972.160310:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762972.160321:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0ead500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074762624:0@lo:400:kworker.0 00000100:00100000:1.0:1789762972.160328:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eadc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074762880:0@lo:400:kworker.0 00000100:00100000:3.0:1789762972.160328:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c962aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074761600:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762972.160337:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f79c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074763008:0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.160367:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762972.160393:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762972.160395:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074761856 00000100:00100000:2.0:1789762972.160400:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074761856 00000100:00100000:0.0:1789762972.160409:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f79c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074763136:0@lo:400:kworker.0 00000100:00100000:3.0:1789762972.160420:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762972.160433:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f79c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074763392:0@lo:400:kworker.0 00000100:00100000:2.0:1789762972.160443:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074761984 00000100:00100000:2.0:1789762972.160447:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074761984 00000100:00100000:0.0:1789762972.160455:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762972.160457:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eaf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074762240:0@lo:400:kworker.0 00000100:00100000:2.0:1789762972.160463:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074761728 00000100:00100000:2.0:1789762972.160466:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074761728 00000100:00100000:1.0:1789762972.160476:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762972.160485:0:14118:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074761600 00000100:00100000:2.0:1789762972.160489:0:14118:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074761600 00000100:00100000:0.0:1789762972.160514:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:0.0:1789762972.160524:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9629500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074761856:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762972.160530:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283 00000100:00100000:1.0:1789762972.160540:0:13091:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:2.0:1789762972.160542:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074761984:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789762972.160545:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 96927744 pending 25559040 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789762972.160556:0:13091:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074761728:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762972.160558:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 5111808 granted 556466176 pending 27262976 free 3859808256 avail 3857711104 00002000:00000024:1.0:1789762972.160594:0:13091:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 5111808 granted 556466176 pending 27262976 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762972.160608:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9629500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074761856:12345-0@lo:13:osp-pre-1-0.0 Request processed in 85us (496us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.160614:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00100000:2.0:1789762972.160634:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074761984:12345-0@lo:13:osp-pre-0-1.0 Request processed in 95us (490us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.160639:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:2.0:1789762972.160646:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283 00000100:00100000:0.0:1789762972.160651:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074761600:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789762972.160654:0:13091:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074761728:12345-0@lo:13:osp-pre-0-0.0 Request processed in 101us (361us total) trans 0 rc 0/0 00002000:00000024:0.0:1789762972.160663:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 96927744 pending 25559040 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789762972.160665:0:13091:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00080000:2.0:1789762972.160668:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762972.160674:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762972.160687:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074762496 00000100:00100000:2.0:1789762972.160691:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074762496 00000100:00100000:1.0:1789762972.160705:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074762368 00000100:00100000:0.0:1789762972.160707:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074761600:12345-0@lo:13:osp-pre-1-1.0 Request processed in 59us (286us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.160712:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074762368 00000100:00100000:0.0:1789762972.160718:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00000100:00100000:2.0:1789762972.160721:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074762880 00000100:00100000:3.0:1789762972.160721:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762972.160727:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074762880 00000100:00100000:0.0:1789762972.160751:0:6411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074762112 00000100:00100000:3.0:1789762972.160752:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074762752:0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.160759:0:6411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074762112 00000100:00100000:1.0:1789762972.160760:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074762624 00000100:00100000:1.0:1789762972.160766:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074762624 00000100:00100000:2.0:1789762972.160767:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074763136 00000100:00100000:2.0:1789762972.160772:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074763136 00000100:00100000:3.0:1789762972.160781:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762972.160799:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f79c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074763264:0@lo:400:kworker.0 00000100:00100000:2.0:1789762972.160802:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074762240 00000100:00100000:2.0:1789762972.160805:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074762240 00000100:00100000:0.0:1789762972.160806:0:6411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:1.0:1789762972.160814:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:0.0:1789762972.160819:0:6411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074762112:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762972.160824:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074762752 00000100:00100000:2.0:1789762972.160830:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074762752 00000100:00100000:3.0:1789762972.160830:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762972.160832:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074762368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762972.160858:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 691 00000100:00100000:2.0:1789762972.160870:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074762496:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.160876:0:6411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:1065be24-cabd-445e-9b93-f5987588951d+24:3427:x1876702074762112:12345-0@lo:400:kworker.0 Request processed in 61us (620us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762972.160879:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074761728:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762972.160886:0:6411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:1.0:1789762972.160903:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074762368:12345-0@lo:400:kworker.0 Request processed in 74us (646us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762972.160907:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c962aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074761600:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762972.160945:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3426:x1876702074762496:12345-0@lo:400:kworker.0 Request processed in 78us (635us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762972.160953:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 691 00000100:00100000:1.0:1789762972.160958:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:0.0:1789762972.160967:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:2.0:1789762972.160978:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 692 00000100:00100000:0.0:1789762972.160981:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074762624:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762972.160991:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e80e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074762880:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762972.161005:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 693 00000100:00100000:1.0:1789762972.161019:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074763136:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.161026:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074762624:12345-0@lo:400:kworker.0 Request processed in 49us (659us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.161033:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00100000:2.0:1789762972.161040:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e80e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3425:x1876702074762880:12345-0@lo:400:kworker.0 Request processed in 52us (648us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762972.161046:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 692 00000100:00100000:2.0:1789762972.161068:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 694 00000100:00100000:0.0:1789762972.161075:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 695 00000100:00100000:2.0:1789762972.161081:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074762240:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762972.161082:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3427:x1876702074763136:12345-0@lo:400:kworker.0 Request processed in 64us (624us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.161091:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074762752:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789762972.161092:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762972.161098:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 693 00000100:00080000:3.0:1789762972.161099:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762972.161145:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3424:x1876702074762240:12345-0@lo:400:kworker.0 Request processed in 68us (427us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.161147:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3424:x1876702074762752:12345-0@lo:400:kworker.0 Request processed in 60us (366us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.161151:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074763392 00000100:00100000:2.0:1789762972.161158:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 694 00000100:00100000:1.0:1789762972.161160:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074763392 00000100:00100000:0.0:1789762972.161165:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 695 00000100:00100000:3.0:1789762972.161191:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f79c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3427:1876702074763136:0@lo:400:kworker.0 00000100:00100000:2.0:1789762972.161201:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eaf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3424:1876702074762240:0@lo:400:kworker.0 00000100:00100000:1.0:1789762972.161207:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074763264 00000100:00100000:1.0:1789762972.161212:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074763264 00000100:00100000:3.0:1789762972.161214:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eadf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:1065be24-cabd-445e-9b93-f5987588951d:3427:1876702074762112:0@lo:400:kworker.0 00000100:00100000:2.0:1789762972.161222:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3424:1876702074762752:0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.161229:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762972.161241:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0ead500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074762624:0@lo:400:kworker.0 00000100:00100000:1.0:1789762972.161254:0:8052:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283 00000100:00100000:2.0:1789762972.161262:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:1.0:1789762972.161270:0:8052:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074763392:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762972.161272:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eae300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074761984:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762972.161277:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074763264:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.161286:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eafb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3426:1876702074762496:0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.161323:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074763008 00000100:00100000:0.0:1789762972.161327:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074763008 00000100:00100000:1.0:1789762972.161335:0:8052:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074763392:12345-0@lo:400:kworker.0 Request processed in 69us (859us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.161346:0:8052:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283 00000100:00100000:2.0:1789762972.161349:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074763264:12345-0@lo:400:kworker.0 Request processed in 75us (521us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762972.161351:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eadc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3425:1876702074762880:0@lo:400:kworker.0 00000100:00100000:2.0:1789762972.161366:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00100000:0.0:1789762972.161373:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 696 00000100:00100000:3.0:1789762972.161374:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f79c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074763392:0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.161385:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074763008:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762972.161392:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eac700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074762368:0@lo:400:kworker.0 00000100:00080000:1.0:1789762972.161395:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762972.161414:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36ca1afb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074761856:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762972.161422:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f79c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074763264:0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.161437:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3426:x1876702074763008:12345-0@lo:400:kworker.0 Request processed in 55us (212us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.161443:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 696 00000100:00100000:0.0:1789762972.161471:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f79c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3426:1876702074763008:0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.189422:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762972.189462:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067565568 00000100:00100000:3.0:1789762972.189468:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067565568 00000100:00100000:3.0:1789762972.189509:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 373 00000100:00100000:3.0:1789762972.189519:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:16306:x1876702067565568:12345-192.168.203.37@tcp:101:dd.0 00000100:00080000:1.0:1789762972.189530:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789762972.189537:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762972.189556:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177ab2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb7a expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762972.189755:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.189809:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067565824 00000100:00100000:1.0:1789762972.189815:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067565824 00000100:00100000:1.0:1789762972.189856:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 374 00000100:00100000:1.0:1789762972.189866:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+16:16307:x1876702067565824:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762972.189876:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762972.189890:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177ab9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.189891:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a800/0xdf5a7ac59f177ac0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb81 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.189974:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177ab9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.189991:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177ab9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762972.190005:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 16777216 00010000:00010000:1.0:1789762972.190007:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177ac7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.190020:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177ac7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.190030:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177ac7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762972.190040:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 16777216 00010000:00010000:1.0:1789762972.190188:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a800/0xdf5a7ac59f177ac0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb81 expref: 21 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.190193:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae400/0xdf5a7ac59f177ab2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb7a expref: 21 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.190230:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177ab9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb7a expref: 19 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.190235:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177ac7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb81 expref: 19 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.190253:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9c37e37a7e00) 00010000:00010000:3.0:1789762972.190259:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789762972.190264:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9c37e37a7e00) 00010000:00010000:3.0:1789762972.190264:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8cae600, rc 0) 00010000:00010000:1.0:1789762972.190270:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789762972.190274:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee81be00, rc 0) 00000100:00100000:1.0:1789762972.190333:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:16307:x1876702067565824:12345-192.168.203.37@tcp:101:dd.0 Request processed in 464us (574us total) trans 4294967459 rc 0/0 00000100:00100000:3.0:1789762972.190333:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:16306:x1876702067565568:12345-192.168.203.37@tcp:101:dd.0 Request processed in 810us (907us total) trans 4294967460 rc 0/0 00000100:00100000:3.0:1789762972.190356:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 373 00000100:00100000:1.0:1789762972.190364:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 374 00000100:00100000:0.0:1789762972.192081:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762972.192137:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067566080 00000100:00100000:0.0:1789762972.192145:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762972.192146:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067566080 00000100:00100000:0.0:1789762972.192174:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.192197:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067565952 00000100:00100000:1.0:1789762972.192204:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067565952 00000100:00100000:2.0:1789762972.192204:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 79 00000100:00100000:2.0:1789762972.192221:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1afb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:2396:x1876702067566080:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762972.192236:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1789762972.192245:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067566336 00000100:00100000:3.0:1789762972.192250:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067566336 00010000:00010000:2.0:1789762972.192255:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae600/0xdf5a7ac59f177ab9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb7a expref: 20 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.192259:0:6404:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 80 00000100:00100000:1.0:1789762972.192276:0:6404:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e70a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:2398:x1876702067565952:12345-192.168.203.37@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1789762972.192293:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 110 00010000:00010000:1.0:1789762972.192294:0:6404:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762972.192296:0:9878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9c37e37a7e00) 00010000:00010000:2.0:1789762972.192303:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762972.192304:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e73100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:16307:x1876702067566336:12345-192.168.203.37@tcp:35:dd.0 00010000:00010000:1.0:1789762972.192316:0:6404:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177ac7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb81 expref: 19 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.192331:0:6404:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9c37e37a7e00) 00010000:00010000:1.0:1789762972.192336:0:6404:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762972.192343:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1afb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:2396:x1876702067566080:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 127us (263us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762972.192353:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 79 00000100:00100000:1.0:1789762972.192369:0:6404:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e70a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:2398:x1876702067565952:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 95us (219us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.192379:0:6404:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 80 00000100:00100000:3.0:1789762972.192519:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e73100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:16307:x1876702067566336:12345-192.168.203.37@tcp:35:dd.0 Request processed in 218us (344us total) trans 4294967461 rc 0/0 00000100:00100000:3.0:1789762972.192526:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 110 00000100:00100000:0.0:1789762972.192560:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.192607:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067566464 00000100:00100000:1.0:1789762972.192613:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067566464 00000100:00100000:1.0:1789762972.192648:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 217 00000100:00100000:1.0:1789762972.192666:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e72a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+37:16306:x1876702067566464:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762972.192675:0:9037:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762972.192688:0:9037:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177adc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb88 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762972.192710:0:9037:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x9:0x0] initial LVB size: 8388608, mtime: 0x6aad9d99, atime: 0x0, ctime: 0x6aad9d99, blocks: 0x1 00010000:00010000:1.0:1789762972.192724:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9:0x0].0x0 since=445s v=(0 0) 00010000:00010000:1.0:1789762972.192729:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762972.192738:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177adc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb88 expref: 38 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.192752:0:9037:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177adc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb88 expref: 38 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.192763:0:9037:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee81aa00, rc 0) 00000100:00100000:1.0:1789762972.192798:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e72a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:16306:x1876702067566464:12345-192.168.203.37@tcp:101:dd.0 Request processed in 141us (239us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.192806:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 217 00000100:00100000:2.0:1789762972.194726:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.194764:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067566592 00000100:00100000:1.0:1789762972.194771:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067566592 00000100:00100000:1.0:1789762972.194804:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 218 00000100:00100000:1.0:1789762972.194812:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+38:16307:x1876702067566592:12345-192.168.203.37@tcp:101:dd.0 00010000:00010000:1.0:1789762972.194821:0:8053:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762972.194829:0:8053:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9de5735/0xdf5a7ac59f177ae3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb8f expref: -99 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.194848:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762972.194852:0:8053:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762972.194859:0:8053:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9de5735/0xdf5a7ac59f177ae3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb8f expref: 39 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.194870:0:8053:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e9de5735/0xdf5a7ac59f177ae3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb8f expref: 39 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.194879:0:8053:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee81b800, rc 0) 00000100:00100000:1.0:1789762972.194912:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:16307:x1876702067566592:12345-192.168.203.37@tcp:101:dd.0 Request processed in 99us (185us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.194949:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 218 00000100:00100000:0.0:1789762972.197155:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762972.197188:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067566720 00000100:00100000:2.0:1789762972.197194:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067566720 00002000:00100000:2.0:1789762972.197227:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x9:0x0] (0->1048575) req@ffff9c37c9e72d80 x1876702067566720/t0(0) o3->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:753/0 lens 488/0 e 0 to 0 dl 1789762983 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762972.197240:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c9e72d80 with x1876702067566720 ext(0->1048575) 00002000:00010000:2.0:1789762972.197245:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c9e72d80 00000100:00100000:2.0:1789762972.197260:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421378559869 time 421378622689 state 0x0 ref 1 00000100:00100000:2.0:1789762972.197271:0:8057:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 9: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37ff763800 gen 0 rate 10 depth 3 token 0 deadline 421378559869 time 421378622689 state 0x0 ref 1 00000100:00100000:2.0:1789762972.197283:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 3 deadline 445444459813 time 445444359813 state 0x1 ref 1 00000100:00100000:2.0:1789762972.197305:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 445444459813 time 445444382882 state 0x0 ref 1 00000100:00100000:2.0:1789762972.197316:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e72d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2395:x1876702067566720:12345-192.168.203.37@tcp:3:dd.0 00000100:00100000:2.0:1789762972.201473:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e72d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2395:x1876702067566720:12345-192.168.203.37@tcp:3:dd.0 Request processed in 4157us (4317us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762972.201481:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1155 00000100:00100000:2.0:1789762972.201488:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37f61cc800 key jobid={dd.0}&nid={192.168.203.37@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff9c37f7b34500 gen 0 rate 10000 depth 3 token 2 deadline 445444459813 time 445444382882 state 0x2 ref 0 00002000:00100000:2.0:1789762972.201497:0:8057:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x9:0x0] (0->1048575) req@ffff9c37c9e72d80 x1876702067566720/t0(0) o3->fa6fc127-0d58-47ca-82d1-2c8ad7b086c1@192.168.203.37@tcp:753/0 lens 488/440 e 0 to 0 dl 1789762983 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762972.201504:0:8057:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff9c37c9e72d80 with x1876702067566720 ext(0->1048575) 00002000:00010000:2.0:1789762972.201507:0:8057:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9c37c9e72d80 00000100:00100000:3.0:1789762972.235947:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762972.235992:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067566976 00000100:00100000:2.0:1789762972.235999:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067566976 00000100:00100000:2.0:1789762972.236037:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 111 00000100:00100000:2.0:1789762972.236045:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:16320:x1876702067566976:12345-192.168.203.37@tcp:35:dd.0 00000100:00100000:2.0:1789762972.236204:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:16320:x1876702067566976:12345-192.168.203.37@tcp:35:dd.0 Request processed in 158us (261us total) trans 4294967462 rc 0/0 00000100:00100000:2.0:1789762972.236216:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 111 00000100:00100000:0.0:1789762972.246470:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.246585:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067567104 00000100:00100000:1.0:1789762972.246592:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067567104 00000100:00100000:1.0:1789762972.246628:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 375 00000100:00100000:1.0:1789762972.246637:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e70700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+18:16321:x1876702067567104:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:1.0:1789762972.246649:0:6419:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762972.246665:0:6419:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177aea lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb96 expref: -99 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.246708:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a800/0xdf5a7ac59f177af1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.246719:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a800/0xdf5a7ac59f177af1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.246731:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a800/0xdf5a7ac59f177af1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762972.246752:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 16384 00000004:00200000:1.0:1789762972.246755:0:6419:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:1.0:1789762972.246766:0:6419:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a800/0xdf5a7ac59f177af1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.246782:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177aea lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb96 expref: 20 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.246794:0:6419:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a800/0xdf5a7ac59f177af1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb96 expref: 19 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.246800:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:1.0:1789762972.246802:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762972.246804:0:6419:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee81a800, rc 0) 00000100:00100000:1.0:1789762972.246836:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e70700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:16321:x1876702067567104:12345-192.168.203.37@tcp:101:rm.0 Request processed in 202us (369us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.246841:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 375 00000100:00100000:0.0:1789762972.247871:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762972.247914:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067567360 00000100:00100000:2.0:1789762972.247937:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067567360 00000100:00100000:2.0:1789762972.247972:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 376 00000100:00100000:2.0:1789762972.247980:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e73800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+19:16321:x1876702067567360:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:2.0:1789762972.247988:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762972.247997:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177af8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cb9d expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762972.248038:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 16384 00010000:00010000:2.0:1789762972.248099:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177af8 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb9d expref: 21 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.248108:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762972.248111:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762972.248114:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030200/0xdf5a7ac59f177af8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb9d expref: 21 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.248123:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c0030200, rc 0) 00000100:00100000:2.0:1789762972.248150:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e73800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:16321:x1876702067567360:12345-192.168.203.37@tcp:101:rm.0 Request processed in 172us (281us total) trans 4294967463 rc 301/301 00000100:00100000:2.0:1789762972.248156:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 376 00000100:00100000:3.0:1789762972.249664:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.249704:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067567488 00000100:00100000:1.0:1789762972.249709:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067567488 00000100:00100000:1.0:1789762972.249738:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 112 00000100:00100000:1.0:1789762972.249744:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c7bbf850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:16321:x1876702067567488:12345-192.168.203.37@tcp:37:rm.0 00000100:00100000:1.0:1789762972.251487:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c7bbf850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:16321:x1876702067567488:12345-192.168.203.37@tcp:37:rm.0 Request processed in 1743us (1823us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.251493:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 112 00000100:00100000:0.0:1789762972.252983:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762972.253016:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067567744 00000100:00100000:2.0:1789762972.253022:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067567744 00000100:00100000:2.0:1789762972.253052:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 113 00000100:00100000:2.0:1789762972.253061:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e72680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+20:16321:x1876702067567744:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:2.0:1789762972.253160:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e72680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:16321:x1876702067567744:12345-192.168.203.37@tcp:35:rm.0 Request processed in 99us (176us total) trans 4294967464 rc 0/0 00000100:00100000:2.0:1789762972.253168:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 113 00000100:00100000:3.0:1789762972.254019:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762972.254060:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067568000 00000100:00100000:0.0:1789762972.254064:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067568000 00000100:00100000:0.0:1789762972.254091:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 377 00000100:00100000:0.0:1789762972.254097:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+21:16321:x1876702067568000:12345-192.168.203.37@tcp:101:rm.0 00010000:00010000:0.0:1789762972.254104:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762972.254113:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177b06 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cba4 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.254150:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6600/0xdf5a7ac59f177b0d lrc: 3/1,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.254158:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6600/0xdf5a7ac59f177b0d lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.254164:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6600/0xdf5a7ac59f177b0d lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762972.254170:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 16384 00010000:00010000:0.0:1789762972.254216:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177b06 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cba4 expref: 24 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.254226:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6600/0xdf5a7ac59f177b0d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cba4 expref: 23 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.254232:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:0.0:1789762972.254234:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1789762972.254236:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d3ee6600, rc 0) 00000100:00100000:0.0:1789762972.254263:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:16321:x1876702067568000:12345-192.168.203.37@tcp:101:rm.0 Request processed in 167us (247us total) trans 4294967465 rc 0/0 00000100:00100000:0.0:1789762972.254268:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 377 00000100:00100000:0.0:1789762972.255094:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762972.255129:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067568128 00000100:00100000:3.0:1789762972.255140:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067568128 00000100:00100000:3.0:1789762972.255168:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 378 00000100:00100000:3.0:1789762972.255174:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e70e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:16321:x1876702067568128:12345-192.168.203.37@tcp:41:rm.0 00000004:00000024:3.0:1789762972.255206:0:13389:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7718567936 avail 7649361920 00000100:00100000:3.0:1789762972.255239:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e70e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:16321:x1876702067568128:12345-192.168.203.37@tcp:41:rm.0 Request processed in 66us (145us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762972.255245:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 378 00000100:00100000:1.0:1789762972.256016:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762972.256050:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067568384 00000100:00100000:0.0:1789762972.256054:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067568384 00000100:00100000:0.0:1789762972.256083:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 379 00000100:00100000:0.0:1789762972.256091:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e82d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:16321:x1876702067568384:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:0.0:1789762972.256117:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1789762972.256130:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177a65 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb34 expref: 23 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.256140:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9c37d3beea00) 00010000:00010000:0.0:1789762972.256149:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a800/0xdf5a7ac59f177af1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb96 expref: 22 pid: 6419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.256155:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:0.0:1789762972.256157:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762972.256166:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177b1b lrc: 3/0,1 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.256173:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177b1b lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.256178:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177b1b lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.256183:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177b22 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.256188:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177b22 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.256193:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177b22 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.256243:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177b29 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.256251:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177b29 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.256256:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177b29 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762972.257019:0:7744:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].2, 64 off8320 00000004:00080000:0.0:1789762972.257028:0:7744:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 445504109413ns, next at 446504107565ns, rc = 0 00000040:00080000:0.0:1789762972.257145:0:7744:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].9, 64 off8768 00000004:00080000:0.0:1789762972.257150:0:7744:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 445504232250ns, next at 446504231270ns, rc = 0 00010000:00010000:0.0:1789762972.257188:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7000/0xdf5a7ac59f177b30 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257196:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7000/0xdf5a7ac59f177b30 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257202:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7000/0xdf5a7ac59f177b30 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257207:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7000/0xdf5a7ac59f177b30 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257211:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7000/0xdf5a7ac59f177b30 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257215:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7000/0xdf5a7ac59f177b30 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257231:0:7744:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762972.257240:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177b29 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257247:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177b29 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257252:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177b29 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257257:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee6000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762972.257261:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177b29 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257267:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177b29 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.257267:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762972.257278:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9c37e37a7100) 00010000:00010000:1.0:1789762972.257278:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7000/0xdf5a7ac59f177b30 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257283:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762972.257288:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177b29 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.257294:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee7000 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762972.257296:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177b29 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.257300:0:6408:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7000/0xdf5a7ac59f177b30 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257309:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177b1b lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.257311:0:6408:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7000/0xdf5a7ac59f177b30 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257323:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177b1b lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257329:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177b1b lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.257331:0:6408:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff9c37e37a7100) 00010000:00010000:0.0:1789762972.257336:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177b1b lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.257337:0:6408:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789762972.257344:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7000/0xdf5a7ac59f177b30 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257345:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee7800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762972.257350:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177b1b lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.257359:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7000/0xdf5a7ac59f177b30 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257361:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177b1b lrc: 5/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257380:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:0.0:1789762972.257382:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762972.257385:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177b1b lrc: 3/0,0 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.257387:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762972.257390:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177b1b lrc: 0/0,0 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.257391:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762972.257395:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00010000:0.0:1789762972.257403:0:7744:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b5857b57 reply state 000000008a0a6fa2 transno 4294967466 ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177b22 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257411:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177b22 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.257415:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x2b0e463d (ffff9c37d3bee800) 00010000:00010000:0.0:1789762972.257417:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1789762972.257419:0:7744:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762972.257453:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e82d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:16321:x1876702067568384:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1362us (1438us total) trans 4294967466 rc 0/0 00000100:00100000:0.0:1789762972.257461:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 379 00000100:00100000:2.0:1789762972.258634:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.258674:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067568640 00000100:00100000:1.0:1789762972.258680:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067568640 00000100:00100000:1.0:1789762972.258714:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 380 00000100:00100000:1.0:1789762972.258724:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+23:16321:x1876702067568640:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:1.0:1789762972.258748:0:11692:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762972.258761:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d14f1400/0xdf5a7ac59f177a2d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb1f expref: 23 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.258781:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9c37e37a7e00) 00010000:00010000:1.0:1789762972.258788:0:11692:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762972.258797:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177b37 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.258807:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177b37 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.258815:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177b37 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.258828:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a600/0xdf5a7ac59f177b3e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.258836:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a600/0xdf5a7ac59f177b3e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.258842:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a600/0xdf5a7ac59f177b3e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.258887:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177b45 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.258895:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177b45 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.258901:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177b45 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789762972.259350:0:11692:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].10, 64 off8832 00000040:00080000:1.0:1789762972.259369:0:11692:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].3, 64 off8384 00010000:00010000:1.0:1789762972.259413:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177b4c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259423:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177b4c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259431:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177b4c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259438:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177b4c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259444:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177b4c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259451:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177b4c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259468:0:11692:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762972.259479:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177b45 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259486:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177b45 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259492:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177b45 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259498:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81ba00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762972.259502:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177b45 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259509:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177b45 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259520:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9c37db743d00) 00010000:00010000:2.0:1789762972.259520:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762972.259527:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762972.259528:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177b4c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259531:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177b45 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.259545:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81ac00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762972.259548:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177b45 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.259550:0:7217:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177b4c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.259585:0:7217:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177b4c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259587:0:11692:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177b37 lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.259598:0:7217:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff9c37db743d00) 00010000:00010000:1.0:1789762972.259600:0:11692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177b37 lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.259604:0:7217:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789762972.259609:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177b4c lrc: 1/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259615:0:11692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177b37 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.259617:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81ac00/0xdf5a7ac59f177b4c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259633:0:11692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177b37 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259641:0:11692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81b000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762972.259642:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762972.259646:0:11692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177b37 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.259646:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762972.259649:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762972.259658:0:11692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177b37 lrc: 5/0,0 mode: CW/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259668:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:1.0:1789762972.259670:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762972.259674:0:11692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177b37 lrc: 3/0,0 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259684:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177b37 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762972.259696:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c446b717 reply state 00000000e0d1c3cd transno 4294967467 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a600/0xdf5a7ac59f177b3e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259705:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a600/0xdf5a7ac59f177b3e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.259711:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x73550ecb (ffff9c37db743700) 00010000:00010000:1.0:1789762972.259714:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1789762972.259716:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762972.259756:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:16321:x1876702067568640:12345-192.168.203.37@tcp:36:rm.0 Request processed in 1032us (1120us total) trans 4294967467 rc 0/0 00000100:00100000:1.0:1789762972.259763:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 380 00000100:00100000:2.0:1789762972.261680:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.261720:0:6422:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067568896 00000100:00100000:1.0:1789762972.261726:0:6422:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067568896 00000100:00100000:1.0:1789762972.261758:0:6422:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 114 00000100:00100000:1.0:1789762972.261768:0:6422:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:16321:x1876702067568896:12345-192.168.203.37@tcp:35:rm.0 00000100:00100000:1.0:1789762972.261868:0:6422:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:16321:x1876702067568896:12345-192.168.203.37@tcp:35:rm.0 Request processed in 102us (188us total) trans 4294967468 rc 0/0 00000100:00100000:1.0:1789762972.261878:0:6422:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 114 00000100:00100000:0.0:1789762972.262644:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.262683:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067569152 00000100:00100000:1.0:1789762972.262688:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067569152 00000100:00100000:1.0:1789762972.262719:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 381 00000100:00100000:1.0:1789762972.262727:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e71180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:16321:x1876702067569152:12345-192.168.203.37@tcp:36:rm.0 00010000:00010000:1.0:1789762972.262750:0:6419:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1789762972.262762:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6600/0xdf5a7ac59f177b0d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cba4 expref: 25 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.262780:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cae000/0xdf5a7ac59f177a11 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb18 expref: 24 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.262791:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743500) 00010000:00010000:1.0:1789762972.262803:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f1779af lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb03 expref: 23 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.262812:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762972.262815:0:6419:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762972.262824:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b200/0xdf5a7ac59f177b53 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.262833:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b200/0xdf5a7ac59f177b53 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.262841:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b200/0xdf5a7ac59f177b53 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.262853:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.262861:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.262867:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263004:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177b61 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263015:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177b61 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263022:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177b61 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762972.263453:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 16384 00010000:00010000:1.0:1789762972.263463:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177b61 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263470:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177b61 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263477:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177b61 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263483:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81be00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762972.263486:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177b61 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263493:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177b61 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263502:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff9c37db743a00) 00010000:00010000:1.0:1789762972.263505:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762972.263508:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177b61 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263515:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81be00/0xdf5a7ac59f177b61 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263531:0:6419:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b200/0xdf5a7ac59f177b53 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263538:0:6419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b200/0xdf5a7ac59f177b53 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263544:0:6419:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b200/0xdf5a7ac59f177b53 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263549:0:6419:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b200/0xdf5a7ac59f177b53 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263555:0:6419:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81b200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762972.263558:0:6419:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b200/0xdf5a7ac59f177b53 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263563:0:6419:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b200/0xdf5a7ac59f177b53 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263586:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762972.263589:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762972.263593:0:6419:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b200/0xdf5a7ac59f177b53 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263599:0:6419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81b200/0xdf5a7ac59f177b53 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762972.263608:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000001a8c6f1 reply state 0000000013c8748b transno 4294967469 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263617:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.263624:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743c00) 00010000:00010000:1.0:1789762972.263627:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1789762972.263629:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762972.263669:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e71180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:16321:x1876702067569152:12345-192.168.203.37@tcp:36:rm.0 Request processed in 942us (1025us total) trans 4294967469 rc 0/0 00000100:00100000:1.0:1789762972.263676:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 381 00010000:00010000:0.0:1789762972.289122:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762972.547809:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762972.547850:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067569280 00000100:00100000:0.0:1789762972.547859:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067569280 00000100:00100000:0.0:1789762972.547910:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 382 00000100:00100000:0.0:1789762972.547947:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:16332:x1876702067569280:12345-192.168.203.37@tcp:101:mkdir.0 00010000:00010000:0.0:1789762972.547968:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762972.547985:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6a00/0xdf5a7ac59f177b68 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cbab expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.548038:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177b6f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.548049:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177b6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.548059:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177b6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762972.548086:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762972.548089:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762972.548103:0:6418:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177b6f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.548128:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6a00/0xdf5a7ac59f177b68 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbab expref: 26 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.548149:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177b6f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbab expref: 25 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.548159:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762972.548163:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762972.548168:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d3ee7a00, rc 0) 00000100:00100000:0.0:1789762972.548226:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:16332:x1876702067569280:12345-192.168.203.37@tcp:101:mkdir.0 Request processed in 308us (423us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.548233:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 382 00000100:00100000:0.0:1789762972.549744:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762972.549782:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067569536 00000100:00100000:3.0:1789762972.549788:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067569536 00000100:00100000:3.0:1789762972.549827:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 383 00000100:00100000:3.0:1789762972.549836:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e73100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:16332:x1876702067569536:12345-192.168.203.37@tcp:36:mkdir.0 00000100:00100000:3.0:1789762972.549976:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074763520:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789762972.550018:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762972.550061:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074763520 00000100:00100000:3.0:1789762972.550063:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7cf6cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762972.550067:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074763520 00000100:00100000:1.0:1789762972.550104:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 457 00000100:00100000:1.0:1789762972.550112:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074763520:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762972.550181:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074763520:12345-0@lo:1000:mdt00_002.0 Request processed in 70us (160us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.550188:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 457 00000100:00100000:3.0:1789762972.550197:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074763520:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789762972.550241:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074763648:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789762972.550264:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762972.550282:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7cf6cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762972.550291:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074763648 00000100:00100000:1.0:1789762972.550295:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074763648 00000100:00100000:1.0:1789762972.550322:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 458 00000100:00100000:1.0:1789762972.550329:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074763648:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762972.550414:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074763648:12345-0@lo:1000:mdt00_002.0 Request processed in 86us (149us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789762972.550421:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 458 00000100:00100000:3.0:1789762972.550428:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074763648:0@lo:1000:mdt00_002.0 00010000:00010000:3.0:1789762972.550461:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.550469:0:6420:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.550477:0:6420:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762972.550488:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074763776:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1789762972.550512:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762972.550530:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7cf6e40 going to sleep for 11 seconds 00000100:00100000:0.0:1789762972.550545:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074763776 00000100:00100000:0.0:1789762972.550549:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074763776 00000100:00100000:0.0:1789762972.550585:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 459 00000100:00100000:0.0:1789762972.550593:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074763776:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1789762972.550601:0:6424:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762972.550610:0:6424:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6c00/0xdf5a7ac59f177b7d lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f177b76 expref: -99 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.550625:0:6424:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177b6f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbab expref: 24 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.550632:0:6424:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6c00/0xdf5a7ac59f177b7d lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f177b76 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.550654:0:6424:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177b6f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbab expref: 24 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.550663:0:6424:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177b6f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbab expref: 24 pid: 6418 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762972.550671:0:6424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9626d80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074763904:192.168.203.37@tcp:104: 00000100:00100000:0.0:1789762972.550715:0:6424:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da49c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789762972.551510:0:6424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9626d80 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6424:1876702074763904:192.168.203.37@tcp:104: 00010000:00010000:0.0:1789762972.551524:0:6424:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6c00/0xdf5a7ac59f177b7d lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177b76 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.551531:0:6424:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762972.551533:0:6424:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762972.551536:0:6424:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6c00/0xdf5a7ac59f177b7d lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177b76 expref: 7 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.551542:0:6424:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d3ee6c00, rc 0) 00000100:00100000:0.0:1789762972.551581:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6420:x1876702074763776:12345-0@lo:101:mdt00_002.0 Request processed in 990us (1068us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.551586:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 459 00000100:00100000:3.0:1789762972.551608:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074763776:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1789762972.551635:0:6420:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.551646:0:6420:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762972.551842:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.551877:0:6406:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067569664 00000100:00100000:1.0:1789762972.551881:0:6406:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067569664 00010000:00010000:1.0:1789762972.551907:0:6406:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177b6f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbab expref: 25 pid: 6418 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.551996:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 92 00000100:00100000:1.0:1789762972.552007:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0c90c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+25:2398:x1876702067569664:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762972.552016:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762972.552022:0:6406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177b6f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbab expref: 25 pid: 6418 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.552033:0:6406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177b6f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbab expref: 25 pid: 6418 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.552047:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177b6f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbab expref: 25 pid: 6418 timeout: 545 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.552059:0:6406:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:1.0:1789762972.552063:0:6406:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6c00/0xdf5a7ac59f177b7d lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177b76 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.552071:0:6406:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6c00/0xdf5a7ac59f177b7d lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f177b76 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.552077:0:6406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6c00/0xdf5a7ac59f177b7d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f177b76 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.552094:0:6406:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6c00/0xdf5a7ac59f177b7d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177b76 expref: 6 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.552101:0:6406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6406:1876702074764032:0@lo:105: 00000100:00100000:1.0:1789762972.552129:0:6406:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762972.552163:0:6403:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074764032 00000100:00100000:0.0:1789762972.552167:0:6403:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074764032 00000100:00100000:1.0:1789762972.552178:0:6406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252f840 going to sleep for 11 seconds 00000100:00100000:0.0:1789762972.552208:0:6403:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 60 00000100:00100000:0.0:1789762972.552217:0:6403:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e83b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6406:x1876702074764032:12345-0@lo:105: 00010000:00010000:0.0:1789762972.552225:0:6403:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.552243:0:6403:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.552250:0:6403:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.552255:0:6403:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.552265:0:6403:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.552271:0:6403:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.552285:0:6403:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c37d8caf400) 00010000:00010000:3.0:1789762972.552310:0:6420:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762972.552318:0:6403:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e83b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6406:x1876702074764032:12345-0@lo:105: Request processed in 102us (186us total) trans 0 rc 0/0 00010000:00010000:3.0:1789762972.552321:0:6420:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762972.552328:0:6403:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 60 00000004:00080000:3.0:1789762972.552342:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1789762972.552357:0:6406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6406:1876702074764032:0@lo:105: 00010000:00010000:1.0:1789762972.552371:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762972.552371:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.552379:0:6420:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.552386:0:6420:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762972.552401:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074764160:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1789762972.552407:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0c90c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+24:2398:x1876702067569664:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 400us (560us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.552415:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 92 00000100:00100000:3.0:1789762972.552446:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762972.552477:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37f7cf6e40 going to sleep for 11 seconds 00000100:00100000:3.0:1789762972.552530:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074764160 00000100:00100000:3.0:1789762972.552535:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074764160 00000100:00100000:3.0:1789762972.552584:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 460 00000100:00100000:3.0:1789762972.552593:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:6420:x1876702074764160:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1789762972.552601:0:13302:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762972.552622:0:13302:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f177b8b lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xdf5a7ac59f177b84 expref: -99 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.552638:0:13302:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 2/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.552649:0:13302:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f177b8b lrc: 4/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xdf5a7ac59f177b84 expref: 8 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.552663:0:13302:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.552715:0:13302:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f177b8b lrc: 3/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177b84 expref: 8 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.552721:0:13302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743c00) 00010000:00010000:3.0:1789762972.552724:0:13302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762972.552727:0:13302:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f177b8b lrc: 3/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177b84 expref: 8 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.552737:0:13302:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8caf800, rc 0) 00000100:00100000:3.0:1789762972.552770:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074764160:12345-0@lo:101:mdt00_002.0 Request processed in 179us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762972.552775:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 460 00000100:00100000:0.0:1789762972.552797:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074764160:0@lo:101:mdt00_002.0 00010000:00010000:0.0:1789762972.552808:0:6420:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 4/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.552814:0:6420:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 4/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762972.562281:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967466 is committed 00000004:00080000:2.0:1789762972.562288:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 16 committed 00000004:00080000:2.0:1789762972.562300:0:6440:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 9 committed 00000001:00080000:2.0:1789762972.562324:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967469 is committed 00010000:00010000:2.0:1789762972.562385:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177a5e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562393:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177a5e lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562400:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177a5e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562405:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030a00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762972.562409:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177a5e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562427:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177a5e lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562436:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x2b0e463d (ffff9c37d3bee800) 00010000:00010000:2.0:1789762972.562438:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762972.562442:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177a5e lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562446:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177a5e lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562458:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177b22 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562462:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177b22 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562467:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177b22 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562471:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee7e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762972.562475:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177b22 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.562476:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074764288:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1789762972.562485:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177b22 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562494:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x2b0e463d (ffff9c37d3bee800) 00010000:00010000:2.0:1789762972.562496:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762972.562499:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177b22 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562503:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177b22 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x2b0e463d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.562519:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789762972.562540:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a600/0xdf5a7ac59f177b3e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.562549:0:3426:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:2.0:1789762972.562551:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a600/0xdf5a7ac59f177b3e lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762972.562553:0:8054:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074764288 00000100:00100000:1.0:1789762972.562558:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074764416:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1789762972.562561:0:8054:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074764288 00010000:00010000:2.0:1789762972.562579:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a600/0xdf5a7ac59f177b3e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562585:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81a600 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762972.562588:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a600/0xdf5a7ac59f177b3e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562594:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a600/0xdf5a7ac59f177b3e lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.562605:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762972.562610:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00010000:00010000:2.0:1789762972.562621:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x73550ecb (ffff9c37db743700) 00010000:00010000:2.0:1789762972.562625:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762972.562628:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a600/0xdf5a7ac59f177b3e lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762972.562629:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074764288:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1789762972.562642:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a600/0xdf5a7ac59f177b3e lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1e:0x0].0x73550ecb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.562648:0:9037:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074764416 00002000:00080000:0.0:1789762972.562652:0:8054:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:10 count 1 00000100:00100000:1.0:1789762972.562653:0:9037:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074764416 00010000:00010000:2.0:1789762972.562659:0:3411:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562665:0:3411:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562670:0:3411:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562674:0:3411:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81a000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762972.562677:0:3411:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762972.562680:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0xa:0x0] initial LVB size: 8388608, mtime: 0x6aad9d9a, atime: 0x0, ctime: 0x6aad9d9a, blocks: 0x1 00010000:00010000:2.0:1789762972.562682:0:3411:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.562687:0:9037:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00010000:00010000:0.0:1789762972.562690:0:8054:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fef354c3/0xdf5a7ac59f177b92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562699:0:3411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743c00) 00000100:00100000:1.0:1789762972.562701:0:9037:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074764416:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1789762972.562705:0:3411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762972.562714:0:3411:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f177b8b lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177b84 expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.562722:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0xa:0x0].0x0 since=445s v=(0 0) 00002000:00080000:1.0:1789762972.562722:0:9037:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:3 count 1 00010000:00010000:2.0:1789762972.562725:0:3411:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f177b8b lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f177b84 expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.562731:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0xa:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762972.562737:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fef354c3/0xdf5a7ac59f177b92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562737:0:3411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f177b8b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xdf5a7ac59f177b84 expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762972.562742:0:9037:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x3:0x0] initial LVB size: 8388608, mtime: 0x6aad9d9a, atime: 0x0, ctime: 0x6aad9d9a, blocks: 0x1 00010000:00010000:0.0:1789762972.562764:0:8054:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fef354c3/0xdf5a7ac59f177b92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.562764:0:9037:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177b99 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562771:0:3411:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f177b8b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177b84 expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.562790:0:8054:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fef354c3/0xdf5a7ac59f177b92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.562792:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=445s v=(0 0) 00000100:00100000:2.0:1789762972.562796:0:3411:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f79c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_001:lustre-MDT0000_UUID:3411:1876702074764544:0@lo:105: 00010000:00010000:1.0:1789762972.562803:0:9037:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762972.562812:0:8054:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fef354c3/0xdf5a7ac59f177b92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.562813:0:9037:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177b99 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762972.562833:0:3411:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789762972.562840:0:9037:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177b99 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.562844:0:8054:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762972.562852:0:9037:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177b99 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762972.562861:0:3411:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d2724e40 going to sleep for 11 seconds 00010000:00010000:1.0:1789762972.562865:0:9037:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177b99 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.562880:0:9037:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:3.0:1789762972.562893:0:6402:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074764544 00000100:00100000:3.0:1789762972.562898:0:6402:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074764544 00000100:00100000:2.0:1789762972.562968:0:6449:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 61 00000100:00100000:2.0:1789762972.562979:0:6449:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:3411:x1876702074764544:12345-0@lo:105: 00010000:00010000:2.0:1789762972.562987:0:6449:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 5/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562994:0:6449:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 5/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.562999:0:6449:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 5/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.563005:0:6449:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 5/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.563010:0:6449:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 6/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.563019:0:6449:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 6/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.563032:0:6449:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c37d8cae400) 00000100:00100000:2.0:1789762972.563074:0:6449:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:3411:x1876702074764544:12345-0@lo:105: Request processed in 99us (245us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762972.563081:0:6449:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 61 00000100:00100000:2.0:1789762972.563121:0:3411:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f79c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_001:lustre-MDT0000_UUID:3411:1876702074764544:0@lo:105: 00010000:00010000:2.0:1789762972.563139:0:3411:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.563145:0:3411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37ee81a000/0xdf5a7ac59f177b5a lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.563674:0:9037:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074764416:12345-0@lo:6:osp-syn-1-0.0 Request processed in 975us (1070us total) trans 4294967434 rc 0/0 00000100:00100000:1.0:1789762972.563682:0:9037:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000004:00080000:3.0:1789762972.563701:0:3426:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37c9626d80/2, rc 0, transno 4294967434 00010000:00010000:1.0:1789762972.563707:0:6408:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1789762972.563714:0:6408:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fef354c3/0xdf5a7ac59f177b92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762972.563715:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074764416:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1789762972.563719:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074764288:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1090us (1199us total) trans 4294968149 rc 0/0 00000004:00080000:3.0:1789762972.563733:0:3426:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37e0e80380/2, rc 0, transno 4294968149 00010000:00010000:1.0:1789762972.563735:0:6408:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee6a00 already unused, calling callback (ffffffffc1554350) 00000100:00100000:0.0:1789762972.563738:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00010000:00010000:1.0:1789762972.563742:0:6408:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fef354c3/0xdf5a7ac59f177b92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762972.563745:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074764288:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1789762972.563757:0:6408:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fef354c3/0xdf5a7ac59f177b92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.563762:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762972.563768:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177b99 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.563770:0:6408:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fef354c3/0xdf5a7ac59f177b92 lrc: 1/0,0 mode: --/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.563781:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762972.563784:0:6408:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fef354c3/0xdf5a7ac59f177b92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.563786:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177b99 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.563809:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177b99 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.563810:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762972.563815:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762972.563819:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762972.563823:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177b99 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.563829:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177b99 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.563852:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762972.563854:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762972.563856:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762972.563866:0:6420:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.563873:0:6420:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762972.563888:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1789762972.563946:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9624700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074764672:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789762972.563988:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762972.564013:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074764672 00000100:00100000:0.0:1789762972.564015:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da4e40 going to sleep for 11 seconds 00000100:00100000:3.0:1789762972.564018:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074764672 00000100:00100000:3.0:1789762972.564046:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:3.0:1789762972.564064:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9627480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074764672:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789762972.564132:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9627480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074764672:12345-0@lo:1000:mdt00_002.0 Request processed in 68us (144us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762972.564138:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00100000:0.0:1789762972.564142:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9624700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074764672:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789762972.564178:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9624700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074764800:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789762972.564207:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762972.564228:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da4e40 going to sleep for 11 seconds 00000100:00100000:1.0:1789762972.564534:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074764800 00000100:00100000:1.0:1789762972.564538:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074764800 00000100:00100000:1.0:1789762972.564576:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:1.0:1789762972.564584:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9624000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074764800:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762972.564728:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9624000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074764800:12345-0@lo:1000:mdt00_002.0 Request processed in 144us (519us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789762972.564734:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:1.0:1789762972.564760:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9624700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074764800:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762972.564817:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e82300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074764928:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762972.564838:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762972.564855:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252f3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789762972.564866:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074764928 00000100:00100000:0.0:1789762972.564871:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074764928 00000100:00100000:0.0:1789762972.564898:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:0.0:1789762972.564905:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e83480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074764928:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789762972.565073:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e83480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074764928:12345-0@lo:1000:mdt00_002.0 Request processed in 167us (231us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789762972.565080:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:1.0:1789762972.565087:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e82300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074764928:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762972.565129:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e82300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074765056:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762972.565156:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762972.565175:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1789762972.565182:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074765056 00000100:00100000:0.0:1789762972.565188:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074765056 00000100:00100000:0.0:1789762972.565221:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:0.0:1789762972.565231:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e83800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074765056:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789762972.565298:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e83800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074765056:12345-0@lo:1000:mdt00_002.0 Request processed in 70us (140us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.565304:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e82300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074765056:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789762972.565308:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00100000:1.0:1789762972.565358:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e82300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074765184:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762972.565382:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762972.565406:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252fcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762972.565420:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074765184 00000100:00100000:3.0:1789762972.565425:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074765184 00000100:00100000:3.0:1789762972.565451:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:3.0:1789762972.565457:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e82680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074765184:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789762972.565522:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e82680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074765184:12345-0@lo:1000:mdt00_002.0 Request processed in 63us (137us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762972.565530:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:1.0:1789762972.565534:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e82300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074765184:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762972.565587:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e82300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074765312:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762972.565605:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762972.565620:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d252fcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762972.565632:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074765312 00000100:00100000:3.0:1789762972.565638:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074765312 00000100:00100000:3.0:1789762972.565672:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 466 00000100:00100000:3.0:1789762972.565680:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e81f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074765312:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789762972.565739:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e81f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6420:x1876702074765312:12345-0@lo:1000:mdt00_002.0 Request processed in 62us (133us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762972.565745:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 466 00000100:00100000:1.0:1789762972.565754:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e82300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074765312:0@lo:1000:mdt00_002.0 00000040:00080000:1.0:1789762972.566723:0:6420:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x2:0x0].15, 984 off43608 00000040:00080000:1.0:1789762972.566856:0:6420:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x2:0x0].15, 1136 off45384 00000004:00080000:1.0:1789762972.566864:0:6420:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c37d252f3c0 with version 32 00000004:00080000:0.0:1789762972.566906:0:7224:0:(osp_trans.c:1336:osp_get_next_request()) ou ffff9c37ee205540 version 32 rpc_version 32 00000020:00100000:0.0:1789762972.566939:0:7224:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1789762972.566945:0:7224:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c37c9625880 x1876702074765568/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2416/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1789762972.566957:0:7224:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9625880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074765568:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1789762972.566987:0:7224:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762972.567005:0:7224:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da4900 going to sleep for 11 seconds 00000100:00100000:3.0:1789762972.567007:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074765568 00000100:00100000:3.0:1789762972.567012:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074765568 00000100:00100000:3.0:1789762972.567039:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:3.0:1789762972.567055:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7224:x1876702074765568:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1789762972.567615:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9627100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7224:x1876702074765568:12345-0@lo:1000:osp_up0-1.0 Request processed in 559us (626us total) trans 4294967470 rc 0/0 00000100:00100000:3.0:1789762972.567624:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:0.0:1789762972.567664:0:7224:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9625880 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7224:1876702074765568:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1789762972.567708:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f177ba0 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.567718:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f177ba0 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.567727:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f177ba0 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762972.567738:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000092ee34c6 reply state 00000000dc007a24 transno 4294967466 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f177ba0 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.567745:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f177ba0 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.567750:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37db743600) 00010000:00010000:1.0:1789762972.567752:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1789762972.567754:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:1.0:1789762972.567757:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x19:0x0]: returning size 16384 00000004:00080000:1.0:1789762972.567761:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1789762972.567767:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.567774:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.567778:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.567794:0:6420:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1789762972.567806:0:6420:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1789762972.567811:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762972.567811:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.567819:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.567821:0:6420:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.567845:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf400 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:0.0:1789762972.567850:0:7217:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762972.567856:0:7217:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.567889:0:7217:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.567894:0:7217:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1789762972.567903:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e73100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:16332:x1876702067569536:12345-192.168.203.37@tcp:36:mkdir.0 Request processed in 18067us (18158us total) trans 4294967466 rc 0/0 00000100:00100000:1.0:1789762972.567909:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 383 00010000:00010000:0.0:1789762972.567946:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762972.567952:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074765696:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762972.567955:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8caf400/0xdf5a7ac59f177b76 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xdf5a7ac59f177b7d expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.567969:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762972.567973:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762972.567977:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762972.567985:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762972.568019:0:7218:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074765696 00000100:00100000:0.0:1789762972.568023:0:7218:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074765696 00000100:00100000:0.0:1789762972.568056:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 81 00000100:00100000:0.0:1789762972.568074:0:7218:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3424:x1876702074765696:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762972.568082:0:7218:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762972.568093:0:7218:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6c00/0xdf5a7ac59f177b7d lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xdf5a7ac59f177b76 expref: 9 pid: 6424 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.568107:0:7218:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762972.568109:0:7218:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762972.568146:0:7218:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3424:x1876702074765696:12345-0@lo:103:ldlm_bl.0 Request processed in 83us (161us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.568152:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 81 00000100:00100000:1.0:1789762972.568178:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074765696:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789762972.575455:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762972.575495:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067569792 00000100:00100000:3.0:1789762972.575499:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067569792 00000100:00100000:3.0:1789762972.575531:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 384 00000100:00100000:3.0:1789762972.575539:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:16333:x1876702067569792:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:3.0:1789762972.575547:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762972.575557:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177ba7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cbb2 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.575615:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177bae lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.575635:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177bae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.575642:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177bae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762972.575660:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762972.575663:0:13389:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:3.0:1789762972.575677:0:13389:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177bae lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.575705:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caea00/0xdf5a7ac59f177ba7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbb2 expref: 7 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.575720:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177bae lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbb2 expref: 6 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.575726:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:3.0:1789762972.575728:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762972.575731:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37d8cafe00, rc 0) 00000100:00100000:3.0:1789762972.575763:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:16333:x1876702067569792:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 225us (310us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762972.575770:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 384 00000100:00100000:3.0:1789762972.577115:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762972.577147:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067570048 00000100:00100000:0.0:1789762972.577152:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067570048 00000100:00100000:0.0:1789762972.577189:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 385 00000100:00100000:0.0:1789762972.577200:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:16333:x1876702067570048:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762972.577211:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762972.577213:0:7744:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1789762972.577226:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8cafe00/0xdf5a7ac59f177bae lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbb2 expref: 6 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.577237:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762972.577240:0:7744:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762972.577247:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6c00/0xdf5a7ac59f177bb5 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cbb9 expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.577306:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6200/0xdf5a7ac59f177bbc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.577314:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6200/0xdf5a7ac59f177bbc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.577320:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6200/0xdf5a7ac59f177bbc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.577331:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.577338:0:7744:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f177b8b lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xdf5a7ac59f177b84 expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.577344:0:7744:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.577374:0:7744:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f177b8b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xdf5a7ac59f177b84 expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762972.577382:0:7744:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9626a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7744:1876702074765824:0@lo:104: 00000100:00100000:0.0:1789762972.577415:0:7744:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762972.577461:0:8068:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1789762972.577465:0:7744:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c3801da4900 going to sleep for 11 seconds 00000100:00100000:1.0:1789762972.577475:0:8068:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074765824 00000100:00100000:1.0:1789762972.577480:0:8068:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074765824 00000100:00100000:1.0:1789762972.577510:0:8068:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 62 00000100:00100000:1.0:1789762972.577518:0:8068:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f7b0d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:7744:x1876702074765824:12345-0@lo:104: 00010000:00010000:1.0:1789762972.577526:0:8068:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762972.577592:0:7744:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9626a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7744:1876702074765824:0@lo:104: 00010000:00010000:1.0:1789762972.577592:0:8068:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1789762972.577603:0:8068:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f7b0d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:7744:x1876702074765824:12345-0@lo:104: Request processed in 84us (185us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.577620:0:8068:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 62 00010000:00010000:0.0:1789762972.577621:0:7744:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.577650:0:6407:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762972.577657:0:6407:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.577666:0:6407:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae400 already unused, calling callback (ffffffffc1f70ee0) 00010000:00010000:3.0:1789762972.577681:0:6407:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762972.577688:0:6407:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.577709:0:6407:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.577714:0:6407:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1789762972.577733:0:6407:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.577746:0:6407:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c37d8cae400/0xdf5a7ac59f177b84 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xdf5a7ac59f177b8b expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762972.577752:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c98b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074765952:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789762972.577762:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762972.577766:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762972.577769:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762972.577784:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762972.577813:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074765952 00000100:00100000:0.0:1789762972.577818:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074765952 00010000:00010000:0.0:1789762972.577834:0:9878:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f177b8b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xdf5a7ac59f177b84 expref: 8 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762972.577857:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 93 00000100:00100000:0.0:1789762972.577865:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3425:x1876702074765952:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789762972.577873:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:1.0:1789762972.588342:0:6440:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967470 is committed 00010000:00010000:0.0:1789762972.590486:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c37d8caf800/0xdf5a7ac59f177b8b lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xdf5a7ac59f177b84 expref: 7 pid: 13302 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.590500:0:9878:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743c00) 00010000:00010000:0.0:1789762972.590505:0:9878:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.590512:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.590517:0:9878:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.590537:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762972.590548:0:7744:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.590557:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762972.590627:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+6:3425:x1876702074765952:12345-0@lo:103:ldlm_bl.0 Request processed in 12761us (12841us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.590636:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 93 00000100:00100000:3.0:1789762972.590670:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c98b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3425:1876702074765952:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789762972.590766:0:7744:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f79c3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7744:1876702074766080:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1789762972.590799:0:7744:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762972.590823:0:7744:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d2724300 going to sleep for 11 seconds 00000100:00100000:0.0:1789762972.590834:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074766080 00000100:00100000:0.0:1789762972.590840:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074766080 00000100:00100000:0.0:1789762972.590871:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:0.0:1789762972.590878:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:7744:x1876702074766080:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1789762972.590962:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:7744:x1876702074766080:12345-0@lo:1000:mdt00_003.0 Request processed in 84us (162us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.590968:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:2.0:1789762972.590977:0:7744:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f79c3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7744:1876702074766080:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1789762972.591020:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177bca lrc: 3/1,0 mode: --/PR res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591029:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177bca lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591035:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177bca lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591045:0:7744:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6200/0xdf5a7ac59f177bbc lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591050:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6200/0xdf5a7ac59f177bbc lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591054:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6200/0xdf5a7ac59f177bbc lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591058:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6200/0xdf5a7ac59f177bbc lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591062:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee6200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762972.591065:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6200/0xdf5a7ac59f177bbc lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591070:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6200/0xdf5a7ac59f177bbc lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591084:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:2.0:1789762972.591087:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762972.591090:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6200/0xdf5a7ac59f177bbc lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591094:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6200/0xdf5a7ac59f177bbc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591101:0:7744:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591104:0:7744:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591108:0:7744:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591112:0:7744:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee7e00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:2.0:1789762972.591113:0:7744:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591117:0:7744:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591121:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37db743c00) 00010000:00010000:2.0:1789762972.591123:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762972.591124:0:7744:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591128:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7e00/0xdf5a7ac59f177bc3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591154:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6c00/0xdf5a7ac59f177bb5 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbb9 expref: 7 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591167:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c37c0030a00/0xdf5a7ac59f177bca lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbb9 expref: 6 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.591173:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37d3bee600) 00010000:00010000:2.0:1789762972.591175:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762972.591176:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c0030a00, rc 0) 00000100:00100000:2.0:1789762972.591212:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:16333:x1876702067570048:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 14016us (14101us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762972.591217:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 385 00000100:00100000:3.0:1789762972.592691:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762972.592724:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067570304 00000100:00100000:0.0:1789762972.592728:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067570304 00000100:00100000:0.0:1789762972.592754:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 386 00000100:00100000:0.0:1789762972.592760:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:16333:x1876702067570304:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762972.592771:0:11692:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762972.592782:0:11692:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d3ee7c00/0xdf5a7ac59f177bd1 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cbc0 expref: -99 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762972.592835:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x19:0x0]: returning size 16384 00010000:00010000:0.0:1789762972.592848:0:11692:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d3ee7c00/0xdf5a7ac59f177bd1 lrc: 1/0,0 mode: --/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbc0 expref: 8 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.592855:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37db743600) 00010000:00010000:0.0:1789762972.592857:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762972.592860:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d3ee7c00/0xdf5a7ac59f177bd1 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbc0 expref: 8 pid: 11692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.592867:0:11692:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d3ee7c00, rc 0) 00000100:00100000:0.0:1789762972.592891:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:16333:x1876702067570304:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 131us (202us total) trans 0 rc 301/301 00000100:00100000:0.0:1789762972.592895:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 386 00000100:00100000:3.0:1789762972.593952:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762972.593980:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067570432 00000100:00100000:0.0:1789762972.593986:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067570432 00000100:00100000:0.0:1789762972.594032:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 387 00000100:00100000:0.0:1789762972.594041:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+6:16333:x1876702067570432:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762972.594048:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762972.594056:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6a00/0xdf5a7ac59f177bd8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cbc7 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.594087:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177bdf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.594098:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177bdf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.594109:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177bdf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762972.594128:0:13389:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762972.594131:0:13389:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:0.0:1789762972.594143:0:13389:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177bdf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.594159:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6a00/0xdf5a7ac59f177bd8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbc7 expref: 8 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.594170:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7a00/0xdf5a7ac59f177bdf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbc7 expref: 7 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.594176:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762972.594178:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762972.594180:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d3ee7a00, rc 0) 00000100:00100000:0.0:1789762972.594208:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:16333:x1876702067570432:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 169us (257us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.594216:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 387 00000100:00100000:3.0:1789762972.595241:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762972.595273:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067570688 00000100:00100000:0.0:1789762972.595278:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067570688 00000100:00100000:0.0:1789762972.595306:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 388 00000100:00100000:0.0:1789762972.595314:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:16333:x1876702067570688:12345-192.168.203.37@tcp:101:lfs.0 00010000:00010000:0.0:1789762972.595323:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762972.595332:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177be6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cbce expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595363:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177bed lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595370:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177bed lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595375:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177bed lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595382:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177bf4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595387:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177bf4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595392:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177bf4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595477:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6e00/0xdf5a7ac59f177bfb lrc: 3/1,0 mode: --/PR res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595484:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6e00/0xdf5a7ac59f177bfb lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595488:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6e00/0xdf5a7ac59f177bfb lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595495:0:6418:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177bed lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595500:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177bed lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595504:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177bed lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595508:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177bed lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595512:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee6000 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762972.595514:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177bed lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595518:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177bed lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595523:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762972.595524:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762972.595526:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177bed lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595530:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177bed lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595536:0:6418:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177bf4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595540:0:6418:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177bf4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595544:0:6418:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177bf4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595548:0:6418:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee7800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762972.595550:0:6418:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177bf4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595554:0:6418:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177bf4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595558:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37e37a7f00) 00010000:00010000:0.0:1789762972.595560:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762972.595561:0:6418:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177bf4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595578:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177bf4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595596:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177be6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbce expref: 9 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595602:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6e00/0xdf5a7ac59f177bfb lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbce expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.595607:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37d3bee600) 00010000:00010000:0.0:1789762972.595609:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762972.595611:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d3ee6e00, rc 0) 00000100:00100000:0.0:1789762972.595635:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:16333:x1876702067570688:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 323us (396us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.595639:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 388 00000100:00100000:3.0:1789762972.596767:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762972.596788:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.596811:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067570944 00000100:00100000:1.0:1789762972.596817:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067570944 00000100:00100000:0.0:1789762972.596830:0:6405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067571072 00000100:00100000:0.0:1789762972.596835:0:6405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067571072 00000100:00100000:1.0:1789762972.596857:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 389 00000100:00100000:1.0:1789762972.596867:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:16333:x1876702067570944:12345-192.168.203.37@tcp:101:lfs.0 00000100:00100000:0.0:1789762972.596874:0:6405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 82 00010000:00010000:1.0:1789762972.596876:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1789762972.596885:0:6405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c98b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2397:x1876702067571072:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762972.596888:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177c02 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cbd5 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.596894:0:6405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762972.596906:0:6405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6e00/0xdf5a7ac59f177bfb lrc: 0/0,0 mode: --/PR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbce expref: 8 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.596914:0:6405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37d3bee600) 00010000:00010000:0.0:1789762972.596935:0:6405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00200000:1.0:1789762972.596948:0:6420:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x19:0x0]: returning size 16384 00000100:00100000:0.0:1789762972.596961:0:6405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067571072:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 78us (170us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.596967:0:6405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 82 00010000:00010000:1.0:1789762972.597034:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177c02 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbd5 expref: 9 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.597053:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37db743600) 00010000:00010000:1.0:1789762972.597055:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789762972.597058:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177c02 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbd5 expref: 9 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.597066:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee81b000, rc 0) 00000100:00100000:1.0:1789762972.597095:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c98b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:16333:x1876702067570944:12345-192.168.203.37@tcp:101:lfs.0 Request processed in 232us (329us total) trans 4294967467 rc 301/301 00000100:00100000:1.0:1789762972.597100:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 389 00000100:00100000:3.0:1789762972.598240:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.598275:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067571328 00000100:00100000:1.0:1789762972.598281:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067571328 00000100:00100000:1.0:1789762972.598309:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 390 00000100:00100000:1.0:1789762972.598316:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9629180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:16333:x1876702067571328:12345-192.168.203.37@tcp:36:lfs.0 00010000:00010000:1.0:1789762972.598350:0:6419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177c10 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.598359:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177c10 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.598366:0:6419:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177c10 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762972.598494:0:6419:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a2db4bbd reply state 00000000a54a2339 transno 4294967468 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177c10 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.598504:0:6419:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177c10 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.598511:0:6419:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37db743600) 00010000:00010000:1.0:1789762972.598514:0:6419:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:1.0:1789762972.598517:0:6419:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:1.0:1789762972.598523:0:6419:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x19:0x0]: returning size 16384 00000100:00100000:1.0:1789762972.598583:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9629180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:16333:x1876702067571328:12345-192.168.203.37@tcp:36:lfs.0 Request processed in 266us (343us total) trans 4294967468 rc 0/0 00000100:00100000:1.0:1789762972.598589:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 390 00000100:00100000:3.0:1789762972.599373:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762972.599437:0:6421:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067571584 00000100:00100000:2.0:1789762972.599454:0:6421:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067571584 00000100:00100000:2.0:1789762972.599484:0:6421:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 115 00000100:00100000:2.0:1789762972.599491:0:6421:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+10:16333:x1876702067571584:12345-192.168.203.37@tcp:35:lfs.0 00000100:00100000:2.0:1789762972.599603:0:6421:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:16333:x1876702067571584:12345-192.168.203.37@tcp:35:lfs.0 Request processed in 110us (231us total) trans 4294967469 rc 0/0 00000100:00100000:2.0:1789762972.599612:0:6421:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 115 00000100:00100000:3.0:1789762972.604755:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762972.604794:0:7744:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067571712 00000100:00100000:2.0:1789762972.604800:0:7744:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067571712 00000100:00100000:2.0:1789762972.604828:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 391 00000100:00100000:2.0:1789762972.604835:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+11:16334:x1876702067571712:12345-192.168.203.37@tcp:101:chmod.0 00010000:00010000:2.0:1789762972.604844:0:7744:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762972.604853:0:7744:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177c17 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cbdc expref: -99 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.604879:0:7744:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f177c1e lrc: 3/1,0 mode: --/PR res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7744 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.604887:0:7744:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f177c1e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.604892:0:7744:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f177c1e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762972.604904:0:7744:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x19:0x0]: returning size 16384 00000004:00200000:2.0:1789762972.604907:0:7744:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000004:00010000:2.0:1789762972.604914:0:7744:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f177c1e lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7744 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.604957:0:7744:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0031a00/0xdf5a7ac59f177c17 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbdc expref: 13 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.604974:0:7744:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f177c1e lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbdc expref: 12 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.604980:0:7744:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37db743600) 00010000:00010000:2.0:1789762972.604982:0:7744:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762972.604984:0:7744:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37c0030800, rc 0) 00000100:00100000:2.0:1789762972.605008:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:16334:x1876702067571712:12345-192.168.203.37@tcp:101:chmod.0 Request processed in 174us (255us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762972.605015:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 391 00000100:00100000:3.0:1789762972.606076:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762972.606108:0:11692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067571968 00000100:00100000:2.0:1789762972.606114:0:11692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067571968 00000100:00100000:2.0:1789762972.606141:0:11692:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 392 00000100:00100000:2.0:1789762972.606148:0:11692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+12:16334:x1876702067571968:12345-192.168.203.37@tcp:36:chmod.0 00010000:00010000:2.0:1789762972.606167:0:11692:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762972.606177:0:11692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030800/0xdf5a7ac59f177c1e lrc: 0/0,0 mode: --/PR res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbdc expref: 12 pid: 7744 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.606187:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37db743600) 00010000:00010000:2.0:1789762972.606190:0:11692:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762972.606199:0:11692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177c25 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.606206:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177c25 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.606211:0:11692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177c25 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762972.606294:0:11692:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cda605cc reply state 000000008a4d51e9 transno 4294967470 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177c25 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.606302:0:11692:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177c25 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.606307:0:11692:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37db743600) 00010000:00010000:2.0:1789762972.606309:0:11692:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:2.0:1789762972.606311:0:11692:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762972.606314:0:11692:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x19:0x0]: returning size 16384 00000100:00100000:2.0:1789762972.606349:0:11692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:16334:x1876702067571968:12345-192.168.203.37@tcp:36:chmod.0 Request processed in 202us (274us total) trans 4294967470 rc 0/0 00000100:00100000:2.0:1789762972.606366:0:11692:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 392 00000100:00100000:3.0:1789762972.629752:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762972.629802:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067572096 00000100:00100000:0.0:1789762972.629808:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067572096 00000100:00100000:0.0:1789762972.629839:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 393 00000100:00100000:0.0:1789762972.629846:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c962d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:16340:x1876702067572096:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:0.0:1789762972.629855:0:13389:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762972.629865:0:13389:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6e00/0xdf5a7ac59f177c2c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cbe3 expref: -99 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.629897:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177c33 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.629906:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177c33 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.629913:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177c33 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.629939:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177c3a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.629949:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177c3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.629954:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177c3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630061:0:13389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177c41 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630069:0:13389:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177c41 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630077:0:13389:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177c41 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762972.630094:0:13389:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177c41 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630105:0:13389:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177c33 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630114:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177c33 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630121:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177c33 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630129:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177c33 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630133:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee7200 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762972.630136:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177c33 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630142:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177c33 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630149:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c37db8acc00) 00010000:00010000:0.0:1789762972.630151:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762972.630154:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177c33 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630158:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7200/0xdf5a7ac59f177c33 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630166:0:13389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177c3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630170:0:13389:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177c3a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630174:0:13389:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177c3a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630178:0:13389:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee7800 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:0.0:1789762972.630179:0:13389:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177c3a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630183:0:13389:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177c3a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630187:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff9c37e37a7f00) 00010000:00010000:0.0:1789762972.630188:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762972.630190:0:13389:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177c3a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630201:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee7800/0xdf5a7ac59f177c3a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630225:0:13389:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6e00/0xdf5a7ac59f177c2c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbe3 expref: 9 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630238:0:13389:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177c41 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbe3 expref: 8 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.630245:0:13389:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37d3bee600) 00010000:00010000:0.0:1789762972.630247:0:13389:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762972.630249:0:13389:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d3ee6000, rc 0) 00000100:00100000:0.0:1789762972.630280:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c962d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:16340:x1876702067572096:12345-192.168.203.37@tcp:101:dd.500 Request processed in 434us (531us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.630286:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 393 00000100:00100000:3.0:1789762972.631484:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762972.631509:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762972.631517:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067572224 00000100:00100000:0.0:1789762972.631522:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067572224 00000100:00100000:0.0:1789762972.631547:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 394 00000100:00100000:0.0:1789762972.631552:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+13:16340:x1876702067572224:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:0.0:1789762972.631560:0:6418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762972.631584:0:6418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d3ee6a00/0xdf5a7ac59f177c48 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cbea expref: -99 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.631606:0:8070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067572352 00010000:00010000:0.0:1789762972.631622:0:6418:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d3ee7c00/0xdf5a7ac59f177c4f lrc: 3/1,0 mode: --/PR res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6418 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.631623:0:8070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067572352 00010000:00010000:0.0:1789762972.631632:0:6418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d3ee7c00/0xdf5a7ac59f177c4f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.631639:0:6418:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d3ee7c00/0xdf5a7ac59f177c4f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x10/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6418 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762972.631654:0:6418:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x19:0x0]: returning size 16384 00000004:00200000:0.0:1789762972.631657:0:6418:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 2 00000100:00100000:1.0:1789762972.631665:0:8070:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 83 00000100:00100000:1.0:1789762972.631673:0:8070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+8:2396:x1876702067572352:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762972.631675:0:6418:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d3ee6a00/0xdf5a7ac59f177c48 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbea expref: 15 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.631684:0:8070:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762972.631690:0:6418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c36d3ee7c00/0xdf5a7ac59f177c4f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x19:0x0].0x0 bits 0x10/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbea expref: 14 pid: 6418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.631696:0:6418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37db743600) 00010000:00010000:1.0:1789762972.631699:0:8070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c36d3ee6000/0xdf5a7ac59f177c41 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbe3 expref: 8 pid: 13389 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762972.631700:0:6418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1789762972.631704:0:6418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c36d3ee7c00, rc 0) 00010000:00010000:1.0:1789762972.631717:0:8070:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37d3bee600) 00010000:00010000:1.0:1789762972.631721:0:8070:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762972.631735:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:16340:x1876702067572224:12345-192.168.203.37@tcp:101:dd.500 Request processed in 182us (251us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.631741:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 394 00000100:00100000:1.0:1789762972.631746:0:8070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067572352:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 72us (234us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.631756:0:8070:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 83 00000100:00100000:3.0:1789762972.632828:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.632870:0:6420:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067572608 00000100:00100000:1.0:1789762972.632876:0:6420:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067572608 00000100:00100000:1.0:1789762972.632906:0:6420:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 395 00000100:00100000:1.0:1789762972.632912:0:6420:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+14:16340:x1876702067572608:12345-192.168.203.37@tcp:101:dd.500 00010000:00010000:1.0:1789762972.632941:0:6420:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762972.632950:0:6420:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177c56 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xb758ac0edef9cbf1 expref: -99 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633008:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ae00/0xdf5a7ac59f177c5d lrc: 3/0,1 mode: --/CW res: [0x240000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633021:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ae00/0xdf5a7ac59f177c5d lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633030:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ae00/0xdf5a7ac59f177c5d lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633044:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177c64 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x19:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633050:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177c64 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x19:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633055:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177c64 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x19:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.633162:0:6420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e83b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074766208:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789762972.633193:0:6420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762972.633219:0:6420:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d14bacc0 going to sleep for 11 seconds 00000100:00100000:3.0:1789762972.633235:0:6423:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074766208 00000100:00100000:3.0:1789762972.633240:0:6423:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074766208 00000100:00100000:3.0:1789762972.633267:0:6423:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:3.0:1789762972.633274:0:6423:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e82300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074766208:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789762972.633327:0:6423:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e82300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6420:x1876702074766208:12345-0@lo:1000:mdt00_002.0 Request processed in 54us (132us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762972.633332:0:6423:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:1.0:1789762972.633357:0:6420:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967470 gen 2 00000004:00080000:1.0:1789762972.633362:0:6420:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967470 committed transno 4294967470 00000100:00100000:1.0:1789762972.633379:0:6420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e83b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6420:1876702074766208:0@lo:1000:mdt00_002.0 00000004:00080000:1.0:1789762972.633626:0:6420:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0xf:0x0], index 0: 0 00010000:00010000:1.0:1789762972.633726:0:6420:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81b600/0xdf5a7ac59f177c6b lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6420 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633734:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81b600/0xdf5a7ac59f177c6b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633739:0:6420:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81b600/0xdf5a7ac59f177c6b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633756:0:6420:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ae00/0xdf5a7ac59f177c5d lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633762:0:6420:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ae00/0xdf5a7ac59f177c5d lrc: 4/0,1 mode: CW/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633766:0:6420:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ae00/0xdf5a7ac59f177c5d lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633770:0:6420:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ae00/0xdf5a7ac59f177c5d lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633773:0:6420:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81ae00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:1.0:1789762972.633775:0:6420:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ae00/0xdf5a7ac59f177c5d lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633779:0:6420:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ae00/0xdf5a7ac59f177c5d lrc: 5/0,0 mode: CW/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633788:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37db743600) 00010000:00010000:1.0:1789762972.633790:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762972.633792:0:6420:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ae00/0xdf5a7ac59f177c5d lrc: 3/0,0 mode: --/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633796:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ae00/0xdf5a7ac59f177c5d lrc: 0/0,0 mode: --/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762972.633803:0:6420:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005a2ac046 reply state 00000000035f4867 transno 4294967471 ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177c64 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x19:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633809:0:6420:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177c64 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633814:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x4009f790 (ffff9c37db743400) 00010000:00010000:1.0:1789762972.633815:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1789762972.633817:0:6420:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789762972.633832:0:6420:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81b000/0xdf5a7ac59f177c56 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbf1 expref: 17 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633840:0:6420:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81b600/0xdf5a7ac59f177c6b lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cbf1 expref: 16 pid: 6420 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.633845:0:6420:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1a:0x0].0x0 (ffff9c37db743e00) 00010000:00010000:1.0:1789762972.633847:0:6420:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789762972.633848:0:6420:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c37ee81b600, rc 0) 00000100:00100000:1.0:1789762972.633876:0:6420:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+17:16340:x1876702067572608:12345-192.168.203.37@tcp:101:dd.500 Request processed in 964us (1050us total) trans 4294967471 rc 0/0 00000100:00100000:1.0:1789762972.633881:0:6420:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 395 00000100:00100000:3.0:1789762972.636972:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.637010:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067572736 00000100:00100000:1.0:1789762972.637017:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067572736 00000100:00100000:1.0:1789762972.637056:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 445348709599 time 445348773394 state 0x0 ref 1 00000100:00100000:1.0:1789762972.637068:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 445548773394 time 445348773394 state 0x1 ref 1 00000100:00100000:1.0:1789762972.637092:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 1 deadline 445548773394 time 445884170517 state 0x0 ref 1 00000100:00100000:1.0:1789762972.637102:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+39:2398:x1876702067572736:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762972.637131:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789762972.637137:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177c79 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.637147:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=445s v=(0 0) 00010000:00010000:1.0:1789762972.637151:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762972.637155:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177c79 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.637164:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177c79 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762972.659429:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968149 is committed 00000001:00080000:0.0:1789762972.659447:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968147 is committed 00000001:00080000:0.0:1789762972.659464:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968148 is committed 00000001:00080000:0.0:1789762972.659472:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968150 is committed 00002000:00010000:1.0:1789762972.659474:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:1.0:1789762972.659480:0:14170:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime from disk: 0 -> 1789762972 00002000:00010000:1.0:1789762972.659483:0:14170:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime from disk: 0 -> 1789762972 00002000:00010000:1.0:1789762972.659486:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:1.0:1789762972.659500:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177c79 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.659514:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177c79 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.659532:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177c79 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.659541:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81b400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762972.659547:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177c79 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.659556:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177c79 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.659591:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177c79 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.659600:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177c79 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.659689:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067572736:12345-192.168.203.37@tcp:4:dd.500 Request processed in 22582us (22711us total) trans 4294968150 rc 0/0 00000100:00100000:1.0:1789762972.659701:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1156 00000100:00100000:1.0:1789762972.659714:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 1 deadline 445548773394 time 445884170517 state 0x2 ref 0 00000100:00100000:3.0:1789762972.679546:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762972.679645:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067572864 00000100:00100000:2.0:1789762972.679654:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067572864 00000100:00100000:2.0:1789762972.679704:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 1 deadline 445548773394 time 445884170517 state 0x0 ref 1 00000100:00100000:2.0:1789762972.679719:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 1 deadline 446084170517 time 445884170517 state 0x1 ref 1 00000100:00100000:2.0:1789762972.679742:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446084170517 time 445926820664 state 0x0 ref 1 00000100:00100000:2.0:1789762972.679753:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067572864:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762972.679793:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 1048576, mtime: 0x6aad9d9c, atime: 0x0, ctime: 0x6aad9d9c, blocks: 0x803 00010000:00010000:2.0:1789762972.679799:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c80 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.679812:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=445s v=(0 0) 00010000:00010000:2.0:1789762972.679815:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762972.679821:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c80 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.679831:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c80 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762972.692286:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968151 is committed 00002000:00010000:2.0:1789762972.692347:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:2.0:1789762972.692352:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:2.0:1789762972.692360:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c80 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.692381:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c80 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.692392:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c80 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.692397:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762972.692400:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c80 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.692406:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c80 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.692419:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c80 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.692424:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c80 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762972.692492:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067572864:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12738us (12953us total) trans 4294968151 rc 0/0 00000100:00100000:2.0:1789762972.692500:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1157 00000100:00100000:2.0:1789762972.692508:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446084170517 time 445926820664 state 0x2 ref 0 00000100:00100000:0.0:1789762972.695143:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762972.695189:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067572992 00000100:00100000:3.0:1789762972.695208:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067572992 00000100:00100000:3.0:1789762972.695239:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446084170517 time 445926820664 state 0x0 ref 1 00000100:00100000:3.0:1789762972.695249:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446126820664 time 445926820664 state 0x1 ref 1 00000100:00100000:2.0:1789762972.879828:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446126820664 time 446126894758 state 0x0 ref 1 00000100:00100000:2.0:1789762972.879852:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9626a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067572992:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762972.879902:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 2097152, mtime: 0x6aad9d9c, atime: 0x0, ctime: 0x6aad9d9c, blocks: 0x1007 00010000:00010000:2.0:1789762972.879909:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c87 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.879941:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=446s v=(0 0) 00010000:00010000:2.0:1789762972.879945:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762972.879950:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c87 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.879962:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c87 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762972.893462:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968152 is committed 00002000:00010000:2.0:1789762972.893515:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:2.0:1789762972.893520:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:2.0:1789762972.893531:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c87 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.893540:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c87 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.893547:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c87 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.893552:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762972.893555:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c87 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.893582:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c87 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.893613:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c87 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.893622:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c87 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762972.893694:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9626a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067572992:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13848us (198548us total) trans 4294968152 rc 0/0 00000100:00100000:2.0:1789762972.893703:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1158 00000100:00100000:2.0:1789762972.893712:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446126820664 time 446126894758 state 0x2 ref 0 00000100:00100000:3.0:1789762972.896373:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762972.896430:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067573120 00000100:00100000:1.0:1789762972.896439:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067573120 00000100:00100000:1.0:1789762972.896496:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446126820664 time 446126894758 state 0x0 ref 1 00000100:00100000:1.0:1789762972.896510:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446326894758 time 446126894758 state 0x1 ref 1 00000100:00100000:3.0:1789762973.079976:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446326894758 time 446327040979 state 0x0 ref 1 00000100:00100000:3.0:1789762973.080019:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067573120:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:1.0:1789762973.080043:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789762973.080072:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 3145728, mtime: 0x6aad9d9c, atime: 0x0, ctime: 0x6aad9d9c, blocks: 0x1807 00010000:00010000:3.0:1789762973.080080:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f177c8e lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.080093:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=446s v=(0 0) 00010000:00010000:3.0:1789762973.080097:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762973.080101:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f177c8e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.080113:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f177c8e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762973.092008:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968153 is committed 00002000:00010000:2.0:1789762973.092029:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:2.0:1789762973.092035:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:2.0:1789762973.092045:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f177c8e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.092055:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f177c8e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.092061:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f177c8e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.092070:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cae400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762973.092074:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f177c8e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.092080:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f177c8e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.092093:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f177c8e lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.092098:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000314ed567/0xdf5a7ac59f177c8e lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762973.092165:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067573120:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12171us (195790us total) trans 4294968153 rc 0/0 00000100:00100000:2.0:1789762973.092173:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1159 00000100:00100000:2.0:1789762973.092182:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446326894758 time 446327040979 state 0x2 ref 0 00000100:00100000:3.0:1789762973.095000:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762973.095031:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067573248 00000100:00100000:2.0:1789762973.095040:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067573248 00000100:00100000:2.0:1789762973.095078:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446326894758 time 446327040979 state 0x0 ref 1 00000100:00100000:2.0:1789762973.095090:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446527040979 time 446327040979 state 0x1 ref 1 00000100:00100000:0.0:1789762973.280081:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446527040979 time 446527143214 state 0x0 ref 1 00000100:00100000:0.0:1789762973.280118:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067573248:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762973.280178:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 4194304, mtime: 0x6aad9d9c, atime: 0x0, ctime: 0x6aad9d9c, blocks: 0x2007 00010000:00010000:0.0:1789762973.280185:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000091b46771/0xdf5a7ac59f177c95 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.280199:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=446s v=(0 0) 00010000:00010000:0.0:1789762973.280202:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762973.280207:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000091b46771/0xdf5a7ac59f177c95 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.280219:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000091b46771/0xdf5a7ac59f177c95 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762973.294023:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 4194304 -> 5242880 00000001:00080000:2.0:1789762973.294026:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968154 is committed 00002000:00010000:0.0:1789762973.294031:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:0.0:1789762973.294046:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000091b46771/0xdf5a7ac59f177c95 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.294060:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000091b46771/0xdf5a7ac59f177c95 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.294070:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000091b46771/0xdf5a7ac59f177c95 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.294079:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee7800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762973.294083:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000091b46771/0xdf5a7ac59f177c95 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.294099:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000091b46771/0xdf5a7ac59f177c95 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.294113:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000091b46771/0xdf5a7ac59f177c95 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.294119:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000091b46771/0xdf5a7ac59f177c95 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762973.294180:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067573248:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14076us (199177us total) trans 4294968154 rc 0/0 00000100:00100000:0.0:1789762973.294190:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1160 00000100:00100000:0.0:1789762973.294202:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446527040979 time 446527143214 state 0x2 ref 0 00000100:00100000:3.0:1789762973.296797:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762973.296838:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067573376 00000100:00100000:2.0:1789762973.296846:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067573376 00000100:00100000:2.0:1789762973.296883:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446527040979 time 446527143214 state 0x0 ref 1 00000100:00100000:2.0:1789762973.296894:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446727143214 time 446527143214 state 0x1 ref 1 00010000:00010000:1.0:1789762973.312056:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762973.480144:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446727143214 time 446727207684 state 0x0 ref 1 00000100:00100000:2.0:1789762973.480174:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067573376:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762973.480228:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 5242880, mtime: 0x6aad9d9c, atime: 0x0, ctime: 0x6aad9d9c, blocks: 0x2807 00010000:00010000:2.0:1789762973.480235:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c9c lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.480249:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=446s v=(0 0) 00010000:00010000:2.0:1789762973.480253:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762973.480257:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c9c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.480268:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c9c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762973.491806:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:2.0:1789762973.491811:0:14170:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime from disk: 1789762972 -> 1789762973 00002000:00010000:2.0:1789762973.491814:0:14170:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime from disk: 1789762972 -> 1789762973 00002000:00010000:2.0:1789762973.491816:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:2.0:1789762973.491824:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c9c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.491835:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c9c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.491841:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c9c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.491846:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762973.491850:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c9c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.491857:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c9c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.491870:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c9c lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.491875:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177c9c lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762973.491964:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2398:x1876702067573376:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11803us (195165us total) trans 4294968155 rc 0/0 00000100:00100000:2.0:1789762973.491973:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1161 00000100:00100000:2.0:1789762973.491982:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446727143214 time 446727207684 state 0x2 ref 0 00000001:00080000:3.0:1789762973.491987:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968155 is committed 00000100:00100000:3.0:1789762973.494320:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762973.494356:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067573504 00000100:00100000:2.0:1789762973.494365:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067573504 00000100:00100000:2.0:1789762973.494403:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446727143214 time 446727207684 state 0x0 ref 1 00000100:00100000:2.0:1789762973.494414:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446927207684 time 446727207684 state 0x1 ref 1 00000100:00100000:3.0:1789762973.680257:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446927207684 time 446927321966 state 0x0 ref 1 00000100:00100000:3.0:1789762973.680303:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067573504:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762973.680355:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 6291456, mtime: 0x6aad9d9d, atime: 0x0, ctime: 0x6aad9d9d, blocks: 0x3007 00010000:00010000:3.0:1789762973.680362:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bec972d4/0xdf5a7ac59f177ca3 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.680375:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=446s v=(0 0) 00010000:00010000:3.0:1789762973.680379:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762973.680383:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bec972d4/0xdf5a7ac59f177ca3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.680395:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bec972d4/0xdf5a7ac59f177ca3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762973.693337:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968156 is committed 00002000:00010000:2.0:1789762973.693368:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:2.0:1789762973.693373:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 12295 -> 14343 00010000:00010000:2.0:1789762973.693383:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bec972d4/0xdf5a7ac59f177ca3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.693392:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bec972d4/0xdf5a7ac59f177ca3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.693398:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bec972d4/0xdf5a7ac59f177ca3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.693403:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caea00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762973.693407:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bec972d4/0xdf5a7ac59f177ca3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.693414:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bec972d4/0xdf5a7ac59f177ca3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.693427:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bec972d4/0xdf5a7ac59f177ca3 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.693432:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bec972d4/0xdf5a7ac59f177ca3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762973.693490:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067573504:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13213us (199169us total) trans 4294968156 rc 0/0 00000100:00100000:2.0:1789762973.693499:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1162 00000100:00100000:2.0:1789762973.693508:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446927207684 time 446927321966 state 0x2 ref 0 00000100:00100000:0.0:1789762973.695724:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762973.695766:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067573632 00000100:00100000:2.0:1789762973.695774:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067573632 00000100:00100000:2.0:1789762973.695811:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 446927207684 time 446927321966 state 0x0 ref 1 00000100:00100000:2.0:1789762973.695822:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447127321966 time 446927321966 state 0x1 ref 1 00000100:00100000:1.0:1789762973.880394:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447127321966 time 447127458904 state 0x0 ref 1 00000100:00100000:1.0:1789762973.880423:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9624e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067573632:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762973.880476:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 7340032, mtime: 0x6aad9d9d, atime: 0x0, ctime: 0x6aad9d9d, blocks: 0x3807 00010000:00010000:1.0:1789762973.880484:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea99889e/0xdf5a7ac59f177caa lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.880497:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=447s v=(0 0) 00010000:00010000:1.0:1789762973.880501:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762973.880506:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea99889e/0xdf5a7ac59f177caa lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.880520:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea99889e/0xdf5a7ac59f177caa lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762973.894096:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968157 is committed 00002000:00010000:2.0:1789762973.894112:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:2.0:1789762973.894118:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:2.0:1789762973.894129:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea99889e/0xdf5a7ac59f177caa lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.894139:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea99889e/0xdf5a7ac59f177caa lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.894145:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea99889e/0xdf5a7ac59f177caa lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.894151:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81ae00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762973.894154:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea99889e/0xdf5a7ac59f177caa lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.894166:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea99889e/0xdf5a7ac59f177caa lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.894179:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea99889e/0xdf5a7ac59f177caa lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.894184:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea99889e/0xdf5a7ac59f177caa lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762973.894257:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9624e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067573632:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13846us (198532us total) trans 4294968157 rc 0/0 00000100:00100000:2.0:1789762973.894264:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1163 00000100:00100000:2.0:1789762973.894273:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447127321966 time 447127458904 state 0x2 ref 0 00000100:00100000:3.0:1789762973.896851:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762973.896887:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067573760 00000100:00100000:2.0:1789762973.896895:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067573760 00000100:00100000:2.0:1789762973.896958:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447127321966 time 447127458904 state 0x0 ref 1 00000100:00100000:2.0:1789762973.896968:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447327458904 time 447127458904 state 0x1 ref 1 00000100:00100000:0.0:1789762974.080518:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447327458904 time 447327584289 state 0x0 ref 1 00000100:00100000:0.0:1789762974.080546:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067573760:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762974.080617:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 8388608, mtime: 0x6aad9d9d, atime: 0x0, ctime: 0x6aad9d9d, blocks: 0x4007 00010000:00010000:0.0:1789762974.080624:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cb1 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.080637:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=447s v=(0 0) 00010000:00010000:0.0:1789762974.080641:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762974.080647:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cb1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.080658:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cb1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762974.092833:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968158 is committed 00002000:00010000:0.0:1789762974.092834:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 8388608 -> 9437184 00002000:00010000:0.0:1789762974.092843:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 16391 -> 18439 00010000:00010000:0.0:1789762974.092856:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cb1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.092866:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cb1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.092873:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cb1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.092878:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762974.092882:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cb1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.092889:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cb1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.092904:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cb1 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.092909:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cb1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762974.092989:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067573760:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12453us (196137us total) trans 4294968158 rc 0/0 00000100:00100000:0.0:1789762974.092997:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1164 00000100:00100000:0.0:1789762974.093007:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447327458904 time 447327584289 state 0x2 ref 0 00000100:00100000:3.0:1789762974.095431:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762974.095472:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067573888 00000100:00100000:2.0:1789762974.095487:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067573888 00000100:00100000:2.0:1789762974.095527:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447327458904 time 447327584289 state 0x0 ref 1 00000100:00100000:2.0:1789762974.095538:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447527584289 time 447327584289 state 0x1 ref 1 00000100:00100000:3.0:1789762974.208138:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074766336:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789762974.208223:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762974.208279:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074766336 00000100:00100000:1.0:1789762974.208287:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074766336 00000100:00100000:1.0:1789762974.208342:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:1.0:1789762974.208351:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074766336:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762974.208384:0:7227:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762974.208437:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074766336:12345-0@lo:41:osp-pre-1-0.0 Request processed in 87us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762974.208444:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00100000:3.0:1789762974.208483:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074766336:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789762974.280638:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447527584289 time 447527704358 state 0x0 ref 1 00000100:00100000:1.0:1789762974.280668:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067573888:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762974.280721:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 9437184, mtime: 0x6aad9d9d, atime: 0x0, ctime: 0x6aad9d9d, blocks: 0x4807 00010000:00010000:1.0:1789762974.280728:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177cb8 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.280743:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=447s v=(0 0) 00010000:00010000:1.0:1789762974.280747:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762974.280751:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177cb8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.280767:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177cb8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762974.293526:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968159 is committed 00002000:00010000:2.0:1789762974.293548:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 9437184 -> 10485760 00002000:00010000:2.0:1789762974.293568:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 18439 -> 20487 00010000:00010000:2.0:1789762974.293581:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177cb8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.293596:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177cb8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.293626:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177cb8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.293635:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81b400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762974.293639:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177cb8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.293646:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177cb8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.293661:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177cb8 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.293667:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012412cd9/0xdf5a7ac59f177cb8 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762974.293743:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067573888:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13087us (198311us total) trans 4294968159 rc 0/0 00000100:00100000:2.0:1789762974.293755:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1165 00000100:00100000:2.0:1789762974.293765:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447527584289 time 447527704358 state 0x2 ref 0 00000100:00100000:3.0:1789762974.296569:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762974.296614:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067574016 00000100:00100000:2.0:1789762974.296623:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067574016 00000100:00100000:2.0:1789762974.296680:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447527584289 time 447527704358 state 0x0 ref 1 00000100:00100000:2.0:1789762974.296705:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447727704358 time 447527704358 state 0x1 ref 1 00010000:00010000:0.0:1789762974.336084:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762974.480752:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447727704358 time 447727816866 state 0x0 ref 1 00000100:00100000:3.0:1789762974.480782:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067574016:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762974.480833:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 10485760, mtime: 0x6aad9d9d, atime: 0x0, ctime: 0x6aad9d9d, blocks: 0x5007 00010000:00010000:3.0:1789762974.480840:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177cbf lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.480853:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=447s v=(0 0) 00010000:00010000:3.0:1789762974.480857:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762974.480861:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177cbf lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.480873:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177cbf lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762974.493566:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968160 is committed 00002000:00010000:2.0:1789762974.493626:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:2.0:1789762974.493631:0:13300:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime from disk: 1789762973 -> 1789762974 00002000:00010000:2.0:1789762974.493633:0:13300:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime from disk: 1789762973 -> 1789762974 00002000:00010000:2.0:1789762974.493635:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:2.0:1789762974.493644:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177cbf lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.493653:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177cbf lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.493662:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177cbf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.493670:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caf200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762974.493675:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177cbf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.493691:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177cbf lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.493712:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177cbf lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.493718:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009194ab4d/0xdf5a7ac59f177cbf lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762974.493779:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067574016:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13010us (197209us total) trans 4294968160 rc 0/0 00000100:00100000:2.0:1789762974.493786:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1166 00000100:00100000:2.0:1789762974.493795:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447727704358 time 447727816866 state 0x2 ref 0 00000100:00100000:3.0:1789762974.496260:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:0.0:1789762974.496314:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067574144 00000100:00100000:0.0:1789762974.496324:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067574144 00000100:00100000:0.0:1789762974.496367:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447727704358 time 447727816866 state 0x0 ref 1 00000100:00100000:0.0:1789762974.496382:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447927816866 time 447727816866 state 0x1 ref 1 00000100:00100000:0.0:1789762974.680880:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447927816866 time 447927923335 state 0x0 ref 1 00000100:00100000:0.0:1789762974.680973:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d921d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067574144:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762974.681096:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 11534336, mtime: 0x6aad9d9e, atime: 0x0, ctime: 0x6aad9d9e, blocks: 0x5807 00010000:00010000:0.0:1789762974.681108:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cc6 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.681148:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=447s v=(0 0) 00010000:00010000:0.0:1789762974.681155:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762974.681164:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cc6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.681219:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cc6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762974.696531:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968161 is committed 00002000:00010000:2.0:1789762974.696607:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 11534336 -> 12582912 00002000:00010000:2.0:1789762974.696614:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 22535 -> 24583 00010000:00010000:2.0:1789762974.696626:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cc6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.696636:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cc6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.696654:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cc6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.696663:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee7600 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762974.696667:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cc6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.696674:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cc6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.696689:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cc6 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.696693:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c98b82d/0xdf5a7ac59f177cc6 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762974.696773:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d921d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067574144:12345-192.168.203.37@tcp:4:dd.500 Request processed in 15866us (200510us total) trans 4294968161 rc 0/0 00000100:00100000:2.0:1789762974.696782:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1167 00000100:00100000:2.0:1789762974.696792:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447927816866 time 447927923335 state 0x2 ref 0 00000100:00100000:0.0:1789762974.699179:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762974.699231:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067574272 00000100:00100000:2.0:1789762974.699240:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067574272 00000100:00100000:2.0:1789762974.699280:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 447927816866 time 447927923335 state 0x0 ref 1 00000100:00100000:2.0:1789762974.699291:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448127923335 time 447927923335 state 0x1 ref 1 00000001:00080000:2.0:1789762974.731017:0:7221:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294967471 is committed 00010000:00010000:3.0:1789762974.731034:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f177ba0 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1789762974.731035:0:7221:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffff9c37d252f240 refcount 2 committed 0 result 0 batchid 15 00000020:00080000:2.0:1789762974.731044:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37d1097c00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffff9c37d252f3c0 00000020:00080000:2.0:1789762974.731051:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x2:0x0].15 00010000:00010000:3.0:1789762974.731052:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f177ba0 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1789762974.731055:0:7221:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff9c37d1097000 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffff9c37f60b1000 00000020:00080000:2.0:1789762974.731062:0:7221:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x2:0x0].15 00010000:00010000:3.0:1789762974.731072:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f177ba0 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731078:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81a400 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762974.731082:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f177ba0 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731090:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f177ba0 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1789762974.731106:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 15 committed batchid 14 00010000:00010000:3.0:1789762974.731107:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37db743600) 00010000:00010000:3.0:1789762974.731118:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762974.731122:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f177ba0 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731127:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81a400/0xdf5a7ac59f177ba0 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731144:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177c10 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731150:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177c10 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731155:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177c10 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731161:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81ba00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762974.731164:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177c10 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731172:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177c10 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731178:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37db743600) 00010000:00010000:3.0:1789762974.731180:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789762974.731183:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177c10 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731187:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81ba00/0xdf5a7ac59f177c10 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6419 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731209:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177c25 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731213:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177c25 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731217:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177c25 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731222:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030c00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762974.731225:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177c25 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731231:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177c25 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731240:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x0 (ffff9c37db743600) 00010000:00010000:3.0:1789762974.731241:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789762974.731243:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177c25 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731250:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37c0030c00/0xdf5a7ac59f177c25 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x19:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11692 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731267:0:3410:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177c64 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731271:0:3410:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177c64 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731276:0:3410:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177c64 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731280:0:3410:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81bc00 already unused, calling callback (ffffffffc1f644f0) 00010000:00010000:3.0:1789762974.731283:0:3410:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177c64 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731289:0:3410:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177c64 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x19:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731296:0:3410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x19:0x0].0x4009f790 (ffff9c37db743400) 00010000:00010000:3.0:1789762974.731298:0:3410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762974.731301:0:3410:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177c64 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x19:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.731305:0:3410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c37ee81bc00/0xdf5a7ac59f177c64 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x19:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6420 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1789762974.731353:0:7233:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 14 00000100:00100000:1.0:1789762974.880982:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448127923335 time 448128045385 state 0x0 ref 1 00000100:00100000:1.0:1789762974.881026:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9625c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067574272:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762974.881078:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 12582912, mtime: 0x6aad9d9e, atime: 0x0, ctime: 0x6aad9d9e, blocks: 0x6007 00010000:00010000:1.0:1789762974.881085:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ccd lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.881099:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=448s v=(0 0) 00010000:00010000:1.0:1789762974.881102:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762974.881107:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ccd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.881119:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ccd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762974.895980:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 12582912 -> 13631488 00002000:00010000:1.0:1789762974.895987:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 24583 -> 26631 00010000:00010000:1.0:1789762974.895997:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ccd lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.896007:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ccd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.896023:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ccd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.896028:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762974.896032:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ccd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.896039:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ccd lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.896052:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ccd lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.896056:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ccd lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762974.896118:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9625c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2397:x1876702067574272:12345-192.168.203.37@tcp:4:dd.500 Request processed in 15116us (196945us total) trans 4294968162 rc 0/0 00000100:00100000:1.0:1789762974.896126:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1168 00000100:00100000:1.0:1789762974.896135:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448127923335 time 448128045385 state 0x2 ref 0 00000001:00080000:1.0:1789762974.896167:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968162 is committed 00000100:00100000:3.0:1789762974.898580:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762974.898628:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067574400 00000100:00100000:2.0:1789762974.898635:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067574400 00000100:00100000:2.0:1789762974.898677:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448127923335 time 448128045385 state 0x0 ref 1 00000100:00100000:2.0:1789762974.898691:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448328045385 time 448128045385 state 0x1 ref 1 00000100:00100000:2.0:1789762975.081043:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448328045385 time 448328107773 state 0x0 ref 1 00000100:00100000:2.0:1789762975.081088:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067574400:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:1.0:1789762975.081125:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762975.081142:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 13631488, mtime: 0x6aad9d9e, atime: 0x0, ctime: 0x6aad9d9e, blocks: 0x6807 00010000:00010000:2.0:1789762975.081150:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177cd4 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.081163:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=448s v=(0 0) 00010000:00010000:2.0:1789762975.081167:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762975.081171:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177cd4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.081183:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177cd4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762975.093811:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968163 is committed 00002000:00010000:1.0:1789762975.093852:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 13631488 -> 14680064 00002000:00010000:1.0:1789762975.093855:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:1.0:1789762975.093866:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177cd4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.093875:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177cd4 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.093885:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177cd4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.093891:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762975.093894:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177cd4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.093900:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177cd4 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.093914:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177cd4 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.093942:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177cd4 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762975.094007:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067574400:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12945us (195425us total) trans 4294968163 rc 0/0 00000100:00100000:1.0:1789762975.094014:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1169 00000100:00100000:1.0:1789762975.094024:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448328045385 time 448328107773 state 0x2 ref 0 00000100:00100000:3.0:1789762975.096737:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762975.096792:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067574528 00000100:00100000:2.0:1789762975.096800:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067574528 00000100:00100000:2.0:1789762975.096840:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448328045385 time 448328107773 state 0x0 ref 1 00000100:00100000:2.0:1789762975.096852:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448528107773 time 448328107773 state 0x1 ref 1 00000100:00100000:1.0:1789762975.232124:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eb1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074766464:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789762975.232204:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762975.232263:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074766464 00000100:00100000:3.0:1789762975.232271:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074766464 00000100:00100000:3.0:1789762975.232323:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:3.0:1789762975.232334:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0e83b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074766464:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789762975.232419:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261516288 avail 2259419136 00000100:00100000:2.0:1789762975.232480:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0e83b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3427:x1876702074766464:12345-0@lo:41:osp-pre-0-1.0 Request processed in 147us (274us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762975.232487:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:3.0:1789762975.232503:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eb1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074766464:0@lo:41:osp-pre-0-1.0 00000001:00080000:3.0:1789762975.246890:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967420 is committed 00000001:00080000:3.0:1789762975.246908:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967434 is committed 00000001:00080000:3.0:1789762975.246913:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967430 is committed 00000001:00080000:3.0:1789762975.246943:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967431 is committed 00000001:00080000:3.0:1789762975.246973:0:9000:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967433 is committed 00000100:00100000:1.0:1789762975.281136:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448528107773 time 448528200796 state 0x0 ref 1 00000100:00100000:1.0:1789762975.281158:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067574528:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762975.281207:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 14680064, mtime: 0x6aad9d9e, atime: 0x0, ctime: 0x6aad9d9e, blocks: 0x7007 00010000:00010000:1.0:1789762975.281214:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177cdb lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.281227:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=448s v=(0 0) 00010000:00010000:1.0:1789762975.281231:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762975.281236:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177cdb lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.281247:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177cdb lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762975.292640:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968164 is committed 00002000:00010000:1.0:1789762975.292656:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 14680064 -> 15728640 00002000:00010000:1.0:1789762975.292661:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:1.0:1789762975.292671:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177cdb lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.292681:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177cdb lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.292687:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177cdb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.292693:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762975.292697:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177cdb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.292704:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177cdb lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.292716:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177cdb lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.292721:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177cdb lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762975.292784:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067574528:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11631us (196044us total) trans 4294968164 rc 0/0 00000100:00100000:1.0:1789762975.292796:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1170 00000100:00100000:1.0:1789762975.292807:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448528107773 time 448528200796 state 0x2 ref 0 00000100:00100000:3.0:1789762975.294981:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762975.295013:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067574656 00000100:00100000:1.0:1789762975.295022:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067574656 00000100:00100000:1.0:1789762975.295061:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448528107773 time 448528200796 state 0x0 ref 1 00000100:00100000:1.0:1789762975.295077:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448728200796 time 448528200796 state 0x1 ref 1 00010000:00010000:0.0:1789762975.360088:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762975.481223:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448728200796 time 448728287426 state 0x0 ref 1 00000100:00100000:0.0:1789762975.481253:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067574656:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:0.0:1789762975.481306:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 15728640, mtime: 0x6aad9d9e, atime: 0x0, ctime: 0x6aad9d9e, blocks: 0x7807 00010000:00010000:0.0:1789762975.481314:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000014919665/0xdf5a7ac59f177ce2 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762975.481327:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=448s v=(0 0) 00010000:00010000:0.0:1789762975.481331:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762975.481336:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000014919665/0xdf5a7ac59f177ce2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762975.481347:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000014919665/0xdf5a7ac59f177ce2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762975.493091:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968165 is committed 00002000:00010000:1.0:1789762975.493110:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 15728640 -> 16777216 00002000:00010000:1.0:1789762975.493115:0:8058:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime from disk: 1789762974 -> 1789762975 00002000:00010000:1.0:1789762975.493117:0:8058:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime from disk: 1789762974 -> 1789762975 00002000:00010000:1.0:1789762975.493120:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 30727 -> 32775 00010000:00010000:1.0:1789762975.493130:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000014919665/0xdf5a7ac59f177ce2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.493139:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000014919665/0xdf5a7ac59f177ce2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.493146:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000014919665/0xdf5a7ac59f177ce2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.493150:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c36d3ee6e00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762975.493154:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000014919665/0xdf5a7ac59f177ce2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.493160:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000014919665/0xdf5a7ac59f177ce2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.493174:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000014919665/0xdf5a7ac59f177ce2 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.493179:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000014919665/0xdf5a7ac59f177ce2 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762975.493241:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067574656:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12001us (198259us total) trans 4294968165 rc 0/0 00000100:00100000:1.0:1789762975.493249:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1171 00000100:00100000:1.0:1789762975.493259:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448728200796 time 448728287426 state 0x2 ref 0 00000100:00100000:3.0:1789762975.495977:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762975.496015:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067574784 00000100:00100000:1.0:1789762975.496024:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067574784 00000100:00100000:1.0:1789762975.496068:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448728200796 time 448728287426 state 0x0 ref 1 00000100:00100000:1.0:1789762975.496081:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448928287426 time 448728287426 state 0x1 ref 1 00000100:00100000:1.0:1789762975.681262:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448928287426 time 448928327261 state 0x0 ref 1 00000100:00100000:1.0:1789762975.681293:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067574784:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762975.681374:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 16777216, mtime: 0x6aad9d9f, atime: 0x0, ctime: 0x6aad9d9f, blocks: 0x8007 00010000:00010000:1.0:1789762975.681385:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ce9 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.681402:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=448s v=(0 0) 00010000:00010000:1.0:1789762975.681407:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762975.681415:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ce9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.681430:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ce9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762975.694612:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968166 is committed 00002000:00010000:1.0:1789762975.694623:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 16777216 -> 17825792 00002000:00010000:1.0:1789762975.694630:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:1.0:1789762975.694641:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ce9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.694655:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ce9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.694662:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ce9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.694667:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762975.694670:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ce9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.694677:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ce9 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.694691:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ce9 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.694696:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177ce9 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762975.694757:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067574784:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13476us (198780us total) trans 4294968166 rc 0/0 00000100:00100000:1.0:1789762975.694765:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1172 00000100:00100000:1.0:1789762975.694775:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448928287426 time 448928327261 state 0x2 ref 0 00000100:00100000:3.0:1789762975.697596:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762975.697638:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067574912 00000100:00100000:1.0:1789762975.697646:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067574912 00000100:00100000:1.0:1789762975.697681:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 448928287426 time 448928327261 state 0x0 ref 1 00000100:00100000:1.0:1789762975.697691:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449128327261 time 448928327261 state 0x1 ref 1 00000100:00100000:2.0:1789762975.881365:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449128327261 time 449128428108 state 0x0 ref 1 00000100:00100000:2.0:1789762975.881400:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067574912:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762975.881460:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 17825792, mtime: 0x6aad9d9f, atime: 0x0, ctime: 0x6aad9d9f, blocks: 0x8807 00010000:00010000:2.0:1789762975.881470:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177cf0 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.881486:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=449s v=(0 0) 00010000:00010000:2.0:1789762975.881491:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762975.881497:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177cf0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.881520:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177cf0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762975.895158:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968167 is committed 00002000:00010000:3.0:1789762975.895159:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 17825792 -> 18874368 00002000:00010000:3.0:1789762975.895166:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 34823 -> 36871 00010000:00010000:3.0:1789762975.895175:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177cf0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.895185:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177cf0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.895191:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177cf0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.895199:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762975.895204:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177cf0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.895220:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177cf0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.895240:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177cf0 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.895249:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177cf0 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762975.895313:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067574912:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13928us (197717us total) trans 4294968167 rc 0/0 00000100:00100000:3.0:1789762975.895322:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1173 00000100:00100000:3.0:1789762975.895331:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449128327261 time 449128428108 state 0x2 ref 0 00000100:00100000:3.0:1789762975.898483:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762975.898571:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067575040 00000100:00100000:1.0:1789762975.898579:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067575040 00000100:00100000:1.0:1789762975.898631:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449128327261 time 449128428108 state 0x0 ref 1 00000100:00100000:1.0:1789762975.898646:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449328428108 time 449128428108 state 0x1 ref 1 00000100:00100000:2.0:1789762976.081513:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449328428108 time 449328574360 state 0x0 ref 1 00000100:00100000:2.0:1789762976.081559:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067575040:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762976.081619:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 18874368, mtime: 0x6aad9d9f, atime: 0x0, ctime: 0x6aad9d9f, blocks: 0x9007 00010000:00010000:2.0:1789762976.081628:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cf7 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762976.081644:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=449s v=(0 0) 00010000:00010000:2.0:1789762976.081649:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762976.081656:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cf7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762976.081672:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cf7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762976.094832:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968168 is committed 00002000:00010000:1.0:1789762976.094888:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:1.0:1789762976.094893:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:1.0:1789762976.094905:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cf7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.094914:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cf7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.094937:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cf7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.094943:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762976.094947:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cf7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.094957:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cf7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.094977:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cf7 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.094986:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cf7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762976.095072:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067575040:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13538us (196585us total) trans 4294968168 rc 0/0 00000100:00100000:1.0:1789762976.095081:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1174 00000100:00100000:1.0:1789762976.095091:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449328428108 time 449328574360 state 0x2 ref 0 00000100:00100000:0.0:1789762976.098701:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762976.098749:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067575168 00000100:00100000:1.0:1789762976.098758:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067575168 00000100:00100000:1.0:1789762976.098797:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449328428108 time 449328574360 state 0x0 ref 1 00000100:00100000:1.0:1789762976.098808:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449528574360 time 449328574360 state 0x1 ref 1 00000100:00100000:3.0:1789762976.200146:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762976.200189:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762976.200206:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762976.200226:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762976.200240:0:6413:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067575296 00000100:00100000:3.0:1789762976.200248:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762976.200250:0:6413:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067575296 00000100:00100000:0.0:1789762976.200260:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067575424 00000100:00100000:1.0:1789762976.200267:0:6419:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067575552 00000100:00100000:0.0:1789762976.200268:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067575424 00000100:00100000:1.0:1789762976.200275:0:6419:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067575552 00000100:00100000:3.0:1789762976.200281:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762976.200308:0:6413:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 401 00000100:00100000:3.0:1789762976.200313:0:6418:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067575808 00000100:00100000:3.0:1789762976.200319:0:6418:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067575808 00000100:00100000:2.0:1789762976.200320:0:6413:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067575296:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762976.200328:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067575936 00000100:00100000:0.0:1789762976.200333:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067575936 00000100:00100000:1.0:1789762976.200338:0:6419:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 697 00000100:00100000:3.0:1789762976.200351:0:6418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 698 00000100:00100000:1.0:1789762976.200353:0:6419:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067575424:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:3.0:1789762976.200364:0:6418:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067575552:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:0.0:1789762976.200371:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 699 00000100:00100000:2.0:1789762976.200386:0:6413:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:0607efc5-93ac-4424-8fa4-2e162e139892+13:2396:x1876702067575296:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 65us (244us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762976.200387:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067575808:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:2.0:1789762976.200402:0:6413:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 401 00000100:00100000:1.0:1789762976.200404:0:6419:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067575424:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 53us (208us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762976.200408:0:6418:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067575552:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 45us (197us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762976.200424:0:6419:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 697 00000100:00100000:3.0:1789762976.200434:0:6418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 698 00000100:00100000:0.0:1789762976.200435:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2396:x1876702067575808:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 52us (183us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762976.200444:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 699 00000100:00100000:2.0:1789762976.200454:0:7744:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 700 00000100:00100000:2.0:1789762976.200465:0:7744:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067575936:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762976.200476:0:8053:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067575680 00000100:00100000:1.0:1789762976.200481:0:8053:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067575680 00000100:00080000:0.0:1789762976.200485:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789762976.200492:0:7744:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:2a924629-ba65-41c7-b7b8-60afcac2e2ea+5:2395:x1876702067575936:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 30us (211us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762976.200493:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762976.200501:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762976.200504:0:7744:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 700 00000100:00100000:1.0:1789762976.200523:0:8053:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 285 00000100:00100000:1.0:1789762976.200533:0:8053:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067575680:12345-192.168.203.37@tcp:400:kworker.0 00000100:00100000:1.0:1789762976.200580:0:8053:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2395:x1876702067575680:12345-192.168.203.37@tcp:400:kworker.0 Request processed in 50us (353us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762976.200586:0:8053:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 285 00000100:00100000:2.0:1789762976.281603:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449528574360 time 449528668668 state 0x0 ref 1 00000100:00100000:2.0:1789762976.281639:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9627800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067575168:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762976.281689:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 19922944, mtime: 0x6aad9d9f, atime: 0x0, ctime: 0x6aad9d9f, blocks: 0x9807 00010000:00010000:2.0:1789762976.281697:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cfe lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762976.281710:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=449s v=(0 0) 00010000:00010000:2.0:1789762976.281714:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762976.281719:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cfe lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762976.281730:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cfe lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762976.294783:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968169 is committed 00002000:00010000:1.0:1789762976.294804:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 19922944 -> 20971520 00002000:00010000:1.0:1789762976.294809:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:1.0:1789762976.294819:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cfe lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.294829:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cfe lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.294835:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cfe lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.294840:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762976.294844:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cfe lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.294851:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cfe lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.294864:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cfe lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.294874:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000969e6129/0xdf5a7ac59f177cfe lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762976.294979:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9627800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067575168:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13347us (196276us total) trans 4294968169 rc 0/0 00000100:00100000:1.0:1789762976.294988:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1175 00000100:00100000:1.0:1789762976.294998:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449528574360 time 449528668668 state 0x2 ref 0 00000100:00100000:0.0:1789762976.298120:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:3.0:1789762976.298163:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067576064 00000100:00100000:3.0:1789762976.298171:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067576064 00000100:00100000:3.0:1789762976.298211:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449528574360 time 449528668668 state 0x0 ref 1 00000100:00100000:3.0:1789762976.298221:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449728668668 time 449528668668 state 0x1 ref 1 00010000:00010000:3.0:1789762976.384106:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762976.481722:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449728668668 time 449728786206 state 0x0 ref 1 00000100:00100000:2.0:1789762976.481755:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9625f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067576064:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762976.481808:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 20971520, mtime: 0x6aad9d9f, atime: 0x0, ctime: 0x6aad9d9f, blocks: 0xa007 00010000:00010000:2.0:1789762976.481815:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177d05 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762976.481828:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=449s v=(0 0) 00010000:00010000:2.0:1789762976.481832:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762976.481837:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177d05 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762976.481848:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177d05 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762976.495220:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968170 is committed 00002000:00010000:1.0:1789762976.495277:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:1.0:1789762976.495283:0:13301:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime from disk: 1789762975 -> 1789762976 00002000:00010000:1.0:1789762976.495287:0:13301:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime from disk: 1789762975 -> 1789762976 00002000:00010000:1.0:1789762976.495290:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:1.0:1789762976.495301:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177d05 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.495314:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177d05 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.495324:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177d05 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.495333:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031a00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762976.495338:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177d05 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.495353:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177d05 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.495368:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177d05 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.495374:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000651c861d/0xdf5a7ac59f177d05 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762976.495437:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9625f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067576064:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13693us (197314us total) trans 4294968170 rc 0/0 00000100:00100000:1.0:1789762976.495447:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1176 00000100:00100000:1.0:1789762976.495458:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449728668668 time 449728786206 state 0x2 ref 0 00000100:00100000:0.0:1789762976.498765:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762976.498807:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067576192 00000100:00100000:1.0:1789762976.498816:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067576192 00000100:00100000:1.0:1789762976.498855:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449728668668 time 449728786206 state 0x0 ref 1 00000100:00100000:1.0:1789762976.498866:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449928786206 time 449728786206 state 0x1 ref 1 00000100:00100000:2.0:1789762976.681816:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449928786206 time 449928880826 state 0x0 ref 1 00000100:00100000:2.0:1789762976.681845:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9626300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067576192:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762976.681899:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 22020096, mtime: 0x6aad9da0, atime: 0x0, ctime: 0x6aad9da0, blocks: 0xa807 00010000:00010000:2.0:1789762976.681907:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177d0c lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762976.681940:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=449s v=(0 0) 00010000:00010000:2.0:1789762976.681944:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762976.681949:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177d0c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762976.681961:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177d0c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762976.695862:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968171 is committed 00002000:00010000:1.0:1789762976.695886:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 22020096 -> 23068672 00002000:00010000:1.0:1789762976.695893:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:1.0:1789762976.695904:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177d0c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.695935:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177d0c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.695942:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177d0c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.695948:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762976.695951:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177d0c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.695958:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177d0c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.695973:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177d0c lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.695985:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000357d9188/0xdf5a7ac59f177d0c lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762976.696060:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9626300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067576192:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14225us (197293us total) trans 4294968171 rc 0/0 00000100:00100000:1.0:1789762976.696068:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1177 00000100:00100000:1.0:1789762976.696078:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449928786206 time 449928880826 state 0x2 ref 0 00000100:00100000:3.0:1789762976.699837:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762976.699880:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067576320 00000100:00100000:1.0:1789762976.699889:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067576320 00000100:00100000:1.0:1789762976.699943:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 449928786206 time 449928880826 state 0x0 ref 1 00000100:00100000:1.0:1789762976.699954:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450128880826 time 449928880826 state 0x1 ref 1 00000100:00100000:1.0:1789762976.881881:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450128880826 time 450128945725 state 0x0 ref 1 00000100:00100000:1.0:1789762976.881910:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067576320:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762976.881979:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 23068672, mtime: 0x6aad9da0, atime: 0x0, ctime: 0x6aad9da0, blocks: 0xb007 00010000:00010000:1.0:1789762976.881987:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d13 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.882000:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=450s v=(0 0) 00010000:00010000:1.0:1789762976.882003:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762976.882008:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d13 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.882021:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d13 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762976.894373:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968172 is committed 00002000:00010000:1.0:1789762976.894389:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:1.0:1789762976.894394:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:1.0:1789762976.894404:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d13 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.894414:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d13 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.894422:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d13 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.894428:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762976.894431:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d13 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.894437:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d13 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.894451:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d13 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.894456:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d13 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762976.894512:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2395:x1876702067576320:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12614us (194674us total) trans 4294968172 rc 0/0 00000100:00100000:1.0:1789762976.894520:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1178 00000100:00100000:1.0:1789762976.894529:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450128880826 time 450128945725 state 0x2 ref 0 00000100:00100000:0.0:1789762976.896908:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762976.896965:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067576448 00000100:00100000:1.0:1789762976.896973:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067576448 00000100:00100000:1.0:1789762976.897016:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450128880826 time 450128945725 state 0x0 ref 1 00000100:00100000:1.0:1789762976.897035:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450328945725 time 450128945725 state 0x1 ref 1 00000100:00100000:2.0:1789762977.082005:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450328945725 time 450329069096 state 0x0 ref 1 00000100:00100000:2.0:1789762977.082052:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9625500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067576448:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:0.0:1789762977.082088:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762977.082107:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 24117248, mtime: 0x6aad9da0, atime: 0x0, ctime: 0x6aad9da0, blocks: 0xb807 00010000:00010000:2.0:1789762977.082115:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d1a lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.082128:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=450s v=(0 0) 00010000:00010000:2.0:1789762977.082132:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762977.082137:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d1a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.082147:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d1a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762977.096208:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968173 is committed 00002000:00010000:1.0:1789762977.096211:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 24117248 -> 25165824 00002000:00010000:1.0:1789762977.096217:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:1.0:1789762977.096228:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d1a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.096238:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d1a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.096245:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d1a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.096250:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762977.096254:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d1a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.096261:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d1a lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.096276:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d1a lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.096281:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d1a lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762977.096357:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9625500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067576448:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14332us (199446us total) trans 4294968173 rc 0/0 00000100:00100000:1.0:1789762977.096365:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1179 00000100:00100000:1.0:1789762977.096375:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450328945725 time 450329069096 state 0x2 ref 0 00000100:00100000:3.0:1789762977.099141:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762977.099190:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067576576 00000100:00100000:1.0:1789762977.099198:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067576576 00000100:00100000:1.0:1789762977.099248:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450328945725 time 450329069096 state 0x0 ref 1 00000100:00100000:1.0:1789762977.099259:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450529069096 time 450329069096 state 0x1 ref 1 00000004:00080000:0.0:1789762977.280077:0:9032:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 450527155707ns, next at 451527152679ns, rc = 0 00000100:00100000:1.0:1789762977.280077:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0e80a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074766848:0@lo:13:osp-pre-1-1.0 00000004:00080000:2.0:1789762977.280111:0:9030:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 450527191610ns, next at 451527188292ns, rc = 0 00000100:00100000:0.0:1789762977.280136:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eb0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074767104:0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.280152:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f79c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074767232:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762977.280184:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762977.280184:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762977.280237:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762977.280246:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074767104 00000100:00100000:3.0:1789762977.280254:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074767104 00000100:00100000:1.0:1789762977.280266:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f79c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074766720:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762977.280276:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d284c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074767744:0@lo:400:kworker.0 00000100:00100000:0.0:1789762977.280279:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d284e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074767616:0@lo:400:kworker.0 00000100:00100000:1.0:1789762977.280296:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762977.280321:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 701 00000100:00100000:3.0:1789762977.280336:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9624380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074767104:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.280338:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762977.280340:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37c9625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074766592:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762977.280342:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762977.280378:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d284ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074768256:0@lo:400:kworker.0 00000100:00100000:0.0:1789762977.280382:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d284d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074768128:0@lo:400:kworker.0 00000100:00100000:1.0:1789762977.280393:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762977.280422:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9624380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3426:x1876702074767104:12345-0@lo:400:kworker.0 Request processed in 91us (240us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762977.280443:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d284df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074767360:0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.280446:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 701 00000100:00100000:2.0:1789762977.280447:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762977.280452:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762977.280475:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074767744 00000100:00100000:3.0:1789762977.280479:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074767744 00000100:00100000:1.0:1789762977.280487:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762977.280496:0:3426:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d284ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074768640:0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.280520:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074768256 00000100:00100000:1.0:1789762977.280522:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d284fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074767872:0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.280526:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074768256 00000100:00100000:0.0:1789762977.280551:0:3426:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762977.280551:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074768128 00000100:00100000:3.0:1789762977.280558:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074768128 00000100:00100000:1.0:1789762977.280578:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762977.280594:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074768640 00000100:00100000:3.0:1789762977.280599:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074767872 00000100:00100000:2.0:1789762977.280600:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074768640 00000100:00100000:0.0:1789762977.280603:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eb0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3426:1876702074767104:0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.280606:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074767872 00000100:00100000:1.0:1789762977.280607:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d284f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074768384:0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.280644:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:1.0:1789762977.280646:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762977.280647:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 702 00000100:00100000:0.0:1789762977.280652:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074766848 00000100:00100000:0.0:1789762977.280658:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074766848 00000100:00100000:2.0:1789762977.280660:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074768256:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.280663:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074767744:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762977.280676:0:3424:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:1.0:1789762977.280683:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37e0eb3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074766976:0@lo:400:kworker.0 00000100:00100000:0.0:1789762977.280703:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074767232 00000100:00100000:0.0:1789762977.280715:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074767232 00000100:00100000:0.0:1789762977.280737:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074766720 00000100:00100000:1.0:1789762977.280738:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762977.280739:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3427:x1876702074768256:12345-0@lo:400:kworker.0 Request processed in 79us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762977.280740:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3427:x1876702074767744:12345-0@lo:400:kworker.0 Request processed in 79us (403us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762977.280741:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074766720 00000100:00100000:0.0:1789762977.280760:0:15371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074766592 00000100:00100000:2.0:1789762977.280761:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:3.0:1789762977.280761:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 702 00000100:00100000:0.0:1789762977.280765:0:15371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074766592 00000100:00100000:1.0:1789762977.280780:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d284e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074767488:0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.280793:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074768384 00000100:00100000:0.0:1789762977.280796:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:3.0:1789762977.280797:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 703 00000100:00100000:2.0:1789762977.280799:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074768384 00000100:00100000:1.0:1789762977.280806:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762977.280818:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d2529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074766848:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789762977.280818:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074768128:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.280826:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:1.0:1789762977.280828:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d284f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074768000:0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.280838:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074768640:12345-0@lo:400:kworker.0 00002000:00000024:0.0:1789762977.280853:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3851419648 avail 3849322496 00000100:00100000:3.0:1789762977.280874:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3426:x1876702074768128:12345-0@lo:400:kworker.0 Request processed in 65us (427us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762977.280892:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 703 00000100:00100000:1.0:1789762977.280893:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762977.280894:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3426:x1876702074768640:12345-0@lo:400:kworker.0 Request processed in 55us (341us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762977.280904:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00100000:3.0:1789762977.280936:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074767488 00000100:00100000:3.0:1789762977.280941:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074767488 00000100:00100000:0.0:1789762977.280944:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d2529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074766848:12345-0@lo:13:osp-pre-1-1.0 Request processed in 105us (734us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762977.280945:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:1.0:1789762977.280956:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d284c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074768512:0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.280959:0:13389:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074768000 00000100:00100000:0.0:1789762977.280963:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:2.0:1789762977.280971:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074768384:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.280975:0:13389:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074768000 00000100:00100000:1.0:1789762977.280991:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762977.281005:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:3.0:1789762977.281011:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 704 00000100:00100000:0.0:1789762977.281017:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37f79c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074767232:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762977.281024:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36cb452680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074767872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.281029:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074768384:12345-0@lo:400:kworker.0 Request processed in 72us (385us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762977.281030:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d284f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074768384:0@lo:400:kworker.0 00002000:00000024:0.0:1789762977.281031:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3828350976 avail 3824156672 00000100:00100000:2.0:1789762977.281045:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00100000:1.0:1789762977.281060:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0e80a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074766848:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789762977.281061:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074768512 00000100:00100000:2.0:1789762977.281067:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074768512 00000100:00100000:2.0:1789762977.281096:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 289 00000100:00100000:3.0:1789762977.281099:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36cb452680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3424:x1876702074767872:12345-0@lo:400:kworker.0 Request processed in 76us (526us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762977.281100:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37f79c3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074767232:12345-0@lo:13:osp-pre-0-0.0 Request processed in 78us (856us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762977.281103:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d284fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3424:1876702074767872:0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.281108:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3424:x1876702074768512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.281119:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 704 00000100:00100000:0.0:1789762977.281129:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00100000:3.0:1789762977.281144:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 705 00000100:00100000:3.0:1789762977.281156:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36ca1adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074767488:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762977.281158:0:15371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:2.0:1789762977.281164:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074768512:12345-0@lo:400:kworker.0 Request processed in 57us (170us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762977.281171:0:14118:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 289 00000100:00100000:0.0:1789762977.281175:0:15371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d184c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074766720:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789762977.281176:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 289 00000100:00100000:1.0:1789762977.281188:0:14118:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9e70a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074766592:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789762977.281191:0:15371:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3828350976 avail 3824156672 00002000:00000024:1.0:1789762977.281202:0:14118:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71368704 pending 0 free 3851419648 avail 3849322496 00000100:00100000:3.0:1789762977.281203:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36ca1adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3424:x1876702074767488:12345-0@lo:400:kworker.0 Request processed in 51us (399us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762977.281212:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 705 00000100:00100000:2.0:1789762977.281219:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d284c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3427:1876702074767744:0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.281231:0:13389:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 706 00000100:00100000:2.0:1789762977.281243:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d284ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3427:1876702074768256:0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.281246:0:13389:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074768000:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762977.281249:0:15371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d184c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3424:x1876702074766720:12345-0@lo:13:osp-pre-0-1.0 Request processed in 75us (949us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762977.281253:0:14118:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9e70a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074766592:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (858us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762977.281274:0:3427:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294968149 gen 2 00000100:00100000:1.0:1789762977.281275:0:14118:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 289 00000004:00080000:2.0:1789762977.281281:0:3427:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37e0e80380, transno 4294968149 00000100:00100000:0.0:1789762977.281288:0:15371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00100000:2.0:1789762977.281303:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f79c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074767232:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762977.281303:0:13389:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3424:x1876702074768000:12345-0@lo:400:kworker.0 Request processed in 61us (413us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762977.281318:0:13389:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 706 00000100:00100000:0.0:1789762977.281322:0:13302:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074767616 00000100:00100000:1.0:1789762977.281325:0:7227:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074767360 00000100:00100000:0.0:1789762977.281327:0:13302:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074767616 00000100:00100000:1.0:1789762977.281342:0:7227:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074767360 00000100:00100000:0.0:1789762977.281377:0:13302:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00080000:2.0:1789762977.281382:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762977.281388:0:7227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00080000:2.0:1789762977.281389:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789762977.281393:0:13302:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37c9624700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074767616:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762977.281394:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762977.281406:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762977.281413:0:7227:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37fdfe3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074767360:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.281436:0:6414:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074766976 00000100:00100000:2.0:1789762977.281441:0:6414:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074766976 00000100:00100000:0.0:1789762977.281457:0:13302:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37c9624700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3426:x1876702074767616:12345-0@lo:400:kworker.0 Request processed in 70us (1120us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762977.281462:0:7227:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37fdfe3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3424:x1876702074767360:12345-0@lo:400:kworker.0 Request processed in 54us (976us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762977.281481:0:7227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00100000:2.0:1789762977.281489:0:6414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:0.0:1789762977.281490:0:13302:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00100000:2.0:1789762977.281507:0:6414:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d15a3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074766976:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762977.281512:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d284e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3424:1876702074767488:0@lo:400:kworker.0 00000100:00100000:1.0:1789762977.281542:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d284f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3424:1876702074768000:0@lo:400:kworker.0 00000100:00100000:0.0:1789762977.281558:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d284e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074767616:0@lo:400:kworker.0 00000100:00100000:1.0:1789762977.281563:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d284c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074768512:0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.281572:0:6414:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d15a3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:1065be24-cabd-445e-9b93-f5987588951d+24:3424:x1876702074766976:12345-0@lo:400:kworker.0 Request processed in 71us (840us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762977.281585:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d284df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074767360:0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.281590:0:6414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:0.0:1789762977.281591:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d284d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3426:1876702074768128:0@lo:400:kworker.0 00000100:00100000:1.0:1789762977.281609:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f79c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3424:1876702074766720:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789762977.281618:0:3426:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967434 gen 2 00000004:00080000:0.0:1789762977.281623:0:3426:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9c37c9626d80, transno 4294967434 00000100:00100000:0.0:1789762977.281637:0:3426:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d284ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3426:1876702074768640:0@lo:400:kworker.0 00000100:00100000:1.0:1789762977.281641:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37c9625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074766592:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762977.281693:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37e0eb3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1065be24-cabd-445e-9b93-f5987588951d:3424:1876702074766976:0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.282061:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450529069096 time 450529129303 state 0x0 ref 1 00000100:00100000:3.0:1789762977.282076:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37e0eb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067576576:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762977.282113:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 25165824, mtime: 0x6aad9da0, atime: 0x0, ctime: 0x6aad9da0, blocks: 0xc007 00010000:00010000:3.0:1789762977.282120:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d21 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.282133:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=450s v=(0 0) 00010000:00010000:3.0:1789762977.282137:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.282141:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d21 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.282152:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d21 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762977.293661:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968174 is committed 00002000:00010000:3.0:1789762977.293661:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:3.0:1789762977.293668:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:3.0:1789762977.293677:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d21 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.293686:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d21 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.293693:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d21 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.293699:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762977.293702:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d21 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.293708:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d21 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.293722:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d21 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.293727:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d21 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762977.293787:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37e0eb1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067576576:12345-192.168.203.37@tcp:4:dd.500 Request processed in 11712us (194646us total) trans 4294968174 rc 0/0 00000100:00100000:3.0:1789762977.293794:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1180 00000100:00100000:3.0:1789762977.293802:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450529069096 time 450529129303 state 0x2 ref 0 00000100:00100000:0.0:1789762977.296790:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762977.296837:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067576704 00000100:00100000:1.0:1789762977.296844:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067576704 00000100:00100000:1.0:1789762977.296876:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450529069096 time 450529129303 state 0x0 ref 1 00000100:00100000:1.0:1789762977.296896:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450729129303 time 450529129303 state 0x1 ref 1 00010000:00010000:1.0:1789762977.408047:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762977.482193:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450729129303 time 450729254301 state 0x0 ref 1 00000100:00100000:2.0:1789762977.482224:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067576704:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762977.482276:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 26214400, mtime: 0x6aad9da0, atime: 0x0, ctime: 0x6aad9da0, blocks: 0xc807 00010000:00010000:2.0:1789762977.482284:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d28 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.482297:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=450s v=(0 0) 00010000:00010000:2.0:1789762977.482301:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762977.482306:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d28 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.482316:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d28 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762977.494153:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968175 is committed 00002000:00010000:1.0:1789762977.494177:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 26214400 -> 27262976 00002000:00010000:1.0:1789762977.494182:0:14171:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime from disk: 1789762976 -> 1789762977 00002000:00010000:1.0:1789762977.494184:0:14171:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime from disk: 1789762976 -> 1789762977 00002000:00010000:1.0:1789762977.494186:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:1.0:1789762977.494196:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d28 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.494211:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d28 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.494221:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d28 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.494229:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762977.494234:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d28 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.494245:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d28 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.494265:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d28 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.494271:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d28 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762977.494335:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067576704:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12121us (197543us total) trans 4294968175 rc 0/0 00000100:00100000:1.0:1789762977.494343:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1181 00000100:00100000:1.0:1789762977.494352:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450729129303 time 450729254301 state 0x2 ref 0 00000100:00100000:3.0:1789762977.497023:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762977.497060:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067576832 00000100:00100000:1.0:1789762977.497069:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067576832 00000100:00100000:1.0:1789762977.497113:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450729129303 time 450729254301 state 0x0 ref 1 00000100:00100000:1.0:1789762977.497124:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450929254301 time 450729254301 state 0x1 ref 1 00000100:00100000:2.0:1789762977.682298:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450929254301 time 450929363848 state 0x0 ref 1 00000100:00100000:2.0:1789762977.682328:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067576832:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762977.682393:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 27262976, mtime: 0x6aad9da1, atime: 0x0, ctime: 0x6aad9da1, blocks: 0xd007 00010000:00010000:2.0:1789762977.682401:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d2f lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.682414:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=450s v=(0 0) 00010000:00010000:2.0:1789762977.682418:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762977.682423:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d2f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.682434:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d2f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762977.695069:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968176 is committed 00002000:00010000:1.0:1789762977.695120:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:1.0:1789762977.695125:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 53255 -> 55303 00010000:00010000:1.0:1789762977.695135:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d2f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.695144:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d2f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.695150:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d2f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.695155:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762977.695159:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d2f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.695165:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d2f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.695180:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d2f lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.695184:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d2f lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762977.695247:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2397:x1876702067576832:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12932us (198225us total) trans 4294968176 rc 0/0 00000100:00100000:1.0:1789762977.695257:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1182 00000100:00100000:1.0:1789762977.695266:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450929254301 time 450929363848 state 0x2 ref 0 00000100:00100000:3.0:1789762977.698064:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762977.698118:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067576960 00000100:00100000:2.0:1789762977.698126:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067576960 00000100:00100000:2.0:1789762977.698164:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 450929254301 time 450929363848 state 0x0 ref 1 00000100:00100000:2.0:1789762977.698174:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451129363848 time 450929363848 state 0x1 ref 1 00000004:00080000:3.0:1789762977.799406:0:18625:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 10 committed 00000004:00080000:3.0:1789762977.799438:0:18625:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 17 committed 00000100:00100000:2.0:1789762977.799556:0:3427:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f79c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074768768:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1789762977.799582:0:3424:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c36d284f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074768896:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789762977.799643:0:3424:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762977.799676:0:3427:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762977.799679:0:9038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074768896 00000100:00100000:2.0:1789762977.799688:0:9038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074768896 00000100:00100000:0.0:1789762977.799720:0:8052:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074768768 00000100:00100000:0.0:1789762977.799726:0:8052:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074768768 00000100:00100000:2.0:1789762977.799737:0:9038:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:2.0:1789762977.799752:0:9038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3424:x1876702074768896:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1789762977.799753:0:8054:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:3.0:1789762977.799766:0:8054:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3427:x1876702074768768:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1789762977.799793:0:8054:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000401:2 count 1 00002000:00080000:2.0:1789762977.799794:0:9038:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000401:9 count 1 00010000:00010000:2.0:1789762977.799817:0:9038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762977.799822:0:8054:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x2:0x0] initial LVB size: 8388608, mtime: 0x6aad9d9a, atime: 0x0, ctime: 0x6aad9d9a, blocks: 0x4007 00010000:00010000:3.0:1789762977.799832:0:8054:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d3d lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.799842:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x2:0x0].0x0 since=451s v=(0 0) 00010000:00010000:2.0:1789762977.799848:0:9038:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177adc lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb88 expref: 7 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.799848:0:8054:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.799856:0:8054:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d3d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.799863:0:9038:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e9de5735/0xdf5a7ac59f177ae3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb8f expref: 7 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.799871:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9:0x0].0x0 since=6s v=(0 0) 00010000:00010000:3.0:1789762977.799874:0:8054:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d3d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.799875:0:9038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762977.799884:0:9038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 5/0,1 mode: --/PW res: [0x280000401:0x9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.799890:0:8054:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d3d lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.799897:0:8054:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d3d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.799911:0:9038:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e9de5735/0xdf5a7ac59f177ae3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb8f expref: 7 pid: 8053 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.799912:0:8054:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762977.799947:0:8069:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762977.799957:0:8069:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d3d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.799964:0:9038:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e9de5735/0xdf5a7ac59f177ae3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb8f expref: 7 pid: 8053 timeout: 551 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762977.799971:0:8069:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8caee00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:0.0:1789762977.799979:0:8069:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d3d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762977.799983:0:9038:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f79c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:9038:1876702074769024:192.168.203.37@tcp:104: 00010000:00010000:0.0:1789762977.799995:0:8069:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d3d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762977.800013:0:8069:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d3d lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762977.800019:0:8069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001995b3c9/0xdf5a7ac59f177d3d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8054 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762977.800033:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762977.800035:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762977.800038:0:8069:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762977.800074:0:9038:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177adc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb88 expref: 7 pid: 9037 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.800086:0:9038:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177adc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb88 expref: 7 pid: 9037 timeout: 551 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762977.800098:0:9038:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37f79c2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:9038:1876702074769152:192.168.203.37@tcp:104: 00000100:00100000:2.0:1789762977.800133:0:9038:0:(client.c:2715:ptlrpc_set_wait()) set ffff9c37d2724900 going to sleep for 11 seconds 00000100:00100000:3.0:1789762977.800495:0:8054:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3427:x1876702074768768:12345-0@lo:6:osp-syn-1-0.0 Request processed in 732us (820us total) trans 4294967435 rc 0/0 00000100:00100000:3.0:1789762977.800501:0:8054:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000004:00080000:2.0:1789762977.800550:0:3427:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c37f79c0a80/2, rc 0, transno 4294967435 00000100:00100000:2.0:1789762977.800556:0:3427:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f79c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3427:1876702074768768:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789762977.801420:0:9038:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f79c0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:9038:1876702074769024:192.168.203.37@tcp:104: 00000100:00100000:2.0:1789762977.801566:0:9038:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37f79c2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:9038:1876702074769152:192.168.203.37@tcp:104: 00010000:00010000:2.0:1789762977.801598:0:9038:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762977.801816:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762977.801841:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762977.801860:0:6404:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067577216 00000100:00100000:2.0:1789762977.801875:0:6404:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067577216 00000100:00100000:0.0:1789762977.801879:0:9878:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067577088 00000100:00100000:0.0:1789762977.801884:0:9878:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067577088 00010000:00010000:2.0:1789762977.801896:0:6404:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177adc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb88 expref: 9 pid: 9037 timeout: 551 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762977.801903:0:9878:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e9de5735/0xdf5a7ac59f177ae3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb8f expref: 9 pid: 8053 timeout: 551 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762977.801971:0:9878:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 94 00000100:00100000:0.0:1789762977.801981:0:9878:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2398:x1876702067577088:12345-192.168.203.37@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1789762977.801988:0:6406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.37@tcp, seq: 95 00010000:00010000:0.0:1789762977.802003:0:9878:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762977.802010:0:9878:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e9de5735/0xdf5a7ac59f177ae3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb8f expref: 9 pid: 8053 timeout: 551 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762977.802011:0:6406:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+9:2396:x1876702067577216:12345-192.168.203.37@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762977.802026:0:6406:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762977.802029:0:9878:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e9de5735/0xdf5a7ac59f177ae3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb8f expref: 9 pid: 8053 timeout: 551 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.802035:0:6406:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177adc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb88 expref: 9 pid: 9037 timeout: 551 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762977.802051:0:9878:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9de5735/0xdf5a7ac59f177ae3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x9:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb8f expref: 9 pid: 8053 timeout: 551 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.802056:0:6406:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177adc lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x9:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb88 expref: 9 pid: 9037 timeout: 551 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762977.802094:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.802096:0:6406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177adc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.37@tcp remote: 0xb758ac0edef9cb88 expref: 8 pid: 9037 timeout: 551 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762977.802099:0:9878:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762977.802109:0:9878:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762977.802116:0:9878:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.802123:0:6406:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762977.802133:0:9878:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762977.802155:0:6406:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2396:x1876702067577216:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 160us (340us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762977.802164:0:9878:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+7:2398:x1876702067577088:12345-192.168.203.37@tcp:103:ldlm_bl.0 Request processed in 183us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762977.802167:0:6406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 95 00010000:00010000:2.0:1789762977.802169:0:9038:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.802182:0:9038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762977.802183:0:9878:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.37@tcp, seq: 94 00010000:00010000:2.0:1789762977.802195:0:9038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.802201:0:9038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.802217:0:9038:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762977.802230:0:7217:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762977.802238:0:7217:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.802248:0:7217:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031400 already unused, calling callback (ffffffffc1554350) 00010000:00010000:3.0:1789762977.802252:0:7217:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.802307:0:7217:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.802317:0:7217:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.802326:0:7217:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd17ec4c/0xdf5a7ac59f177d36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 9038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.802338:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762977.802339:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762977.802340:0:7217:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762977.802729:0:9038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3424:x1876702074768896:12345-0@lo:6:osp-syn-0-0.0 Request processed in 2981us (3083us total) trans 4294968177 rc 0/0 00000100:00100000:2.0:1789762977.802739:0:9038:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000004:00080000:1.0:1789762977.802751:0:3424:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff9c36d284f800/2, rc 0, transno 4294968177 00000100:00100000:1.0:1789762977.802757:0:3424:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c36d284f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3424:1876702074768896:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762977.882351:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451129363848 time 451129415603 state 0x0 ref 1 00000100:00100000:2.0:1789762977.882374:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067576960:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762977.882436:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 28311552, mtime: 0x6aad9da1, atime: 0x0, ctime: 0x6aad9da1, blocks: 0xd807 00010000:00010000:2.0:1789762977.882445:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d44 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.882471:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=451s v=(0 0) 00010000:00010000:2.0:1789762977.882477:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762977.882485:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d44 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.882499:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d44 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762977.901021:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968178 is committed 00000001:00080000:2.0:1789762977.901045:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968177 is committed 00002000:00010000:1.0:1789762977.901104:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 28311552 -> 29360128 00002000:00010000:1.0:1789762977.901109:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 55303 -> 57351 00010000:00010000:1.0:1789762977.901122:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d44 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.901135:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d44 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.901142:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d44 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.901148:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0031c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762977.901151:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d44 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.901159:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d44 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.901173:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d44 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.901178:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024f15b62/0xdf5a7ac59f177d44 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762977.901247:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067576960:12345-192.168.203.37@tcp:4:dd.500 Request processed in 18880us (203182us total) trans 4294968178 rc 0/0 00000100:00100000:1.0:1789762977.901255:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1183 00000100:00100000:1.0:1789762977.901264:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451129363848 time 451129415603 state 0x2 ref 0 00000100:00100000:1.0:1789762977.903764:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762977.903816:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067577344 00000100:00100000:2.0:1789762977.903823:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067577344 00000100:00100000:2.0:1789762977.903862:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451129363848 time 451129415603 state 0x0 ref 1 00000100:00100000:2.0:1789762977.903873:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451329415603 time 451129415603 state 0x1 ref 1 00000100:00100000:2.0:1789762978.082424:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451329415603 time 451329488526 state 0x0 ref 1 00000100:00100000:2.0:1789762978.082467:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067577344:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762978.082517:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 29360128, mtime: 0x6aad9da1, atime: 0x0, ctime: 0x6aad9da1, blocks: 0xe007 00010000:00010000:2.0:1789762978.082540:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d4b lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.082555:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=451s v=(0 0) 00010000:00010000:2.0:1789762978.082558:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762978.082563:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d4b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.082579:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d4b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762978.099306:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968179 is committed 00002000:00010000:2.0:1789762978.099382:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 29360128 -> 30408704 00002000:00010000:2.0:1789762978.099387:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 57351 -> 59399 00010000:00010000:2.0:1789762978.099398:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d4b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.099407:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d4b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.099414:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d4b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.099419:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762978.099423:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d4b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.099429:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d4b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.099456:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d4b lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.099461:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d4b lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762978.099545:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067577344:12345-192.168.203.37@tcp:4:dd.500 Request processed in 17091us (195778us total) trans 4294968179 rc 0/0 00000100:00100000:2.0:1789762978.099554:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1184 00000100:00100000:2.0:1789762978.099564:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451329415603 time 451329488526 state 0x2 ref 0 00000100:00100000:1.0:1789762978.102404:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762978.102484:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067577472 00000100:00100000:2.0:1789762978.102493:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067577472 00000100:00100000:2.0:1789762978.102553:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451329415603 time 451329488526 state 0x0 ref 1 00000100:00100000:2.0:1789762978.102565:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451529488526 time 451329488526 state 0x1 ref 1 00000100:00100000:1.0:1789762978.282665:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451529488526 time 451529691862 state 0x0 ref 1 00000100:00100000:1.0:1789762978.282727:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067577472:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762978.282862:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 30408704, mtime: 0x6aad9da1, atime: 0x0, ctime: 0x6aad9da1, blocks: 0xe807 00010000:00010000:1.0:1789762978.282879:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177d52 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.282906:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=451s v=(0 0) 00010000:00010000:1.0:1789762978.282915:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762978.282989:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177d52 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.283030:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177d52 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762978.303677:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968180 is committed 00002000:00010000:2.0:1789762978.303688:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 30408704 -> 31457280 00002000:00010000:2.0:1789762978.303695:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 59399 -> 61447 00010000:00010000:2.0:1789762978.303711:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177d52 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.303725:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177d52 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.303735:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177d52 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.303743:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81a800 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762978.303748:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177d52 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.303758:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177d52 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.303789:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177d52 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.303796:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db3b2aeb/0xdf5a7ac59f177d52 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762978.303899:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067577472:12345-192.168.203.37@tcp:4:dd.500 Request processed in 21197us (201497us total) trans 4294968180 rc 0/0 00000100:00100000:2.0:1789762978.303911:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1185 00000100:00100000:2.0:1789762978.304002:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451529488526 time 451529691862 state 0x2 ref 0 00000100:00100000:0.0:1789762978.307620:0:3418:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762978.307741:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067577600 00000100:00100000:1.0:1789762978.307782:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067577600 00000100:00100000:1.0:1789762978.307860:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451529488526 time 451529691862 state 0x0 ref 1 00000100:00100000:1.0:1789762978.307880:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451729691862 time 451529691862 state 0x1 ref 1 00010000:00010000:3.0:1789762978.432103:0:6407:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762978.482812:0:8057:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451729691862 time 451729864203 state 0x0 ref 1 00000100:00100000:1.0:1789762978.482857:0:8057:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067577600:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762978.482988:0:8057:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 31457280, mtime: 0x6aad9da1, atime: 0x0, ctime: 0x6aad9da1, blocks: 0xf007 00010000:00010000:1.0:1789762978.483001:0:8057:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dee62040/0xdf5a7ac59f177d59 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.483027:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=451s v=(0 0) 00010000:00010000:1.0:1789762978.483034:0:8057:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762978.483042:0:8057:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dee62040/0xdf5a7ac59f177d59 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.483066:0:8057:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dee62040/0xdf5a7ac59f177d59 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762978.499253:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968181 is committed 00002000:00010000:2.0:1789762978.499283:0:8057:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 31457280 -> 32505856 00002000:00010000:2.0:1789762978.499288:0:8057:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime from disk: 1789762977 -> 1789762978 00002000:00010000:2.0:1789762978.499290:0:8057:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime from disk: 1789762977 -> 1789762978 00002000:00010000:2.0:1789762978.499292:0:8057:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 61447 -> 63495 00010000:00010000:2.0:1789762978.499303:0:8057:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dee62040/0xdf5a7ac59f177d59 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.499325:0:8057:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dee62040/0xdf5a7ac59f177d59 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.499333:0:8057:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dee62040/0xdf5a7ac59f177d59 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.499339:0:8057:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81b200 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762978.499343:0:8057:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dee62040/0xdf5a7ac59f177d59 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.499349:0:8057:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dee62040/0xdf5a7ac59f177d59 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.499364:0:8057:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dee62040/0xdf5a7ac59f177d59 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.499369:0:8057:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dee62040/0xdf5a7ac59f177d59 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8057 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762978.499448:0:8057:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067577600:12345-192.168.203.37@tcp:4:dd.500 Request processed in 16610us (191830us total) trans 4294968181 rc 0/0 00000100:00100000:2.0:1789762978.499457:0:8057:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1186 00000100:00100000:2.0:1789762978.499467:0:8057:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451729691862 time 451729864203 state 0x2 ref 0 00000100:00100000:1.0:1789762978.503159:0:3419:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762978.503213:0:8059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067577728 00000100:00100000:2.0:1789762978.503224:0:8059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067577728 00000100:00100000:2.0:1789762978.503272:0:8059:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451729691862 time 451729864203 state 0x0 ref 1 00000100:00100000:2.0:1789762978.503283:0:8059:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451929864203 time 451729864203 state 0x1 ref 1 00000100:00100000:3.0:1789762978.682913:0:13300:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451929864203 time 451929977041 state 0x0 ref 1 00000100:00100000:3.0:1789762978.682960:0:13300:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067577728:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:3.0:1789762978.683022:0:13300:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 32505856, mtime: 0x6aad9da2, atime: 0x0, ctime: 0x6aad9da2, blocks: 0xf807 00010000:00010000:3.0:1789762978.683032:0:13300:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f177d60 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.683049:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=451s v=(0 0) 00010000:00010000:3.0:1789762978.683054:0:13300:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762978.683059:0:13300:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f177d60 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.683071:0:13300:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f177d60 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762978.697652:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968182 is committed 00002000:00010000:2.0:1789762978.697654:0:13300:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 32505856 -> 33554432 00002000:00010000:2.0:1789762978.697661:0:13300:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 63495 -> 65543 00010000:00010000:2.0:1789762978.697672:0:13300:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f177d60 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.697683:0:13300:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f177d60 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.697689:0:13300:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f177d60 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.697694:0:13300:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37d8cafa00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762978.697697:0:13300:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f177d60 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.697704:0:13300:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f177d60 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.697720:0:13300:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f177d60 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.697725:0:13300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000082bba5db/0xdf5a7ac59f177d60 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13300 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762978.697797:0:13300:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067577728:12345-192.168.203.37@tcp:4:dd.500 Request processed in 14846us (194647us total) trans 4294968182 rc 0/0 00000100:00100000:2.0:1789762978.697806:0:13300:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1187 00000100:00100000:2.0:1789762978.697816:0:13300:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451929864203 time 451929977041 state 0x2 ref 0 00000100:00100000:3.0:1789762978.700265:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762978.700305:0:8058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067577856 00000100:00100000:2.0:1789762978.700313:0:8058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067577856 00000100:00100000:2.0:1789762978.700353:0:8058:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 451929864203 time 451929977041 state 0x0 ref 1 00000100:00100000:2.0:1789762978.700365:0:8058:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452129977041 time 451929977041 state 0x1 ref 1 00000100:00100000:2.0:1789762978.883008:0:14171:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452129977041 time 452130068705 state 0x0 ref 1 00000100:00100000:2.0:1789762978.883041:0:14171:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067577856:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:2.0:1789762978.883104:0:14171:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 33554432, mtime: 0x6aad9da2, atime: 0x0, ctime: 0x6aad9da2, blocks: 0x10007 00010000:00010000:2.0:1789762978.883114:0:14171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d67 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.883130:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=452s v=(0 0) 00010000:00010000:2.0:1789762978.883135:0:14171:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762978.883142:0:14171:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d67 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.883159:0:14171:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d67 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762978.896337:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968183 is committed 00002000:00010000:1.0:1789762978.896355:0:14171:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:1.0:1789762978.896360:0:14171:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 65543 -> 67591 00010000:00010000:1.0:1789762978.896372:0:14171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d67 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.896382:0:14171:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d67 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.896389:0:14171:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d67 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.896394:0:14171:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37c0030c00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762978.896398:0:14171:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d67 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.896405:0:14171:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d67 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.896419:0:14171:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d67 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.896424:0:14171:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e880956/0xdf5a7ac59f177d67 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14171 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762978.896497:0:14171:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067577856:12345-192.168.203.37@tcp:4:dd.500 Request processed in 13468us (196230us total) trans 4294968183 rc 0/0 00000100:00100000:1.0:1789762978.896506:0:14171:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1188 00000100:00100000:1.0:1789762978.896526:0:14171:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452129977041 time 452130068705 state 0x2 ref 0 00000100:00100000:3.0:1789762978.899192:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762978.899250:0:8057:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067577984 00000100:00100000:2.0:1789762978.899259:0:8057:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067577984 00000100:00100000:2.0:1789762978.899302:0:8057:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452129977041 time 452130068705 state 0x0 ref 1 00000100:00100000:2.0:1789762978.899320:0:8057:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452330068705 time 452130068705 state 0x1 ref 1 00000100:00100000:1.0:1789762979.083116:0:14170:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452330068705 time 452330167973 state 0x0 ref 1 00000100:00100000:1.0:1789762979.083158:0:14170:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067577984:12345-192.168.203.37@tcp:4:dd.500 00000100:00080000:2.0:1789762979.083196:0:6415:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789762979.083213:0:14170:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 34603008, mtime: 0x6aad9da2, atime: 0x0, ctime: 0x6aad9da2, blocks: 0x10807 00010000:00010000:1.0:1789762979.083221:0:14170:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d6e lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.083235:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=452s v=(0 0) 00010000:00010000:1.0:1789762979.083238:0:14170:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762979.083242:0:14170:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d6e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.083254:0:14170:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d6e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762979.095913:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968184 is committed 00002000:00010000:1.0:1789762979.095961:0:14170:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:1.0:1789762979.095966:0:14170:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 67591 -> 69639 00010000:00010000:1.0:1789762979.095977:0:14170:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d6e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.095988:0:14170:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d6e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.095995:0:14170:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d6e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.096000:0:14170:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81aa00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762979.096003:0:14170:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d6e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.096011:0:14170:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d6e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.096025:0:14170:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d6e lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.096030:0:14170:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d6e lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14170 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762979.096109:0:14170:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067577984:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12969us (196913us total) trans 4294968184 rc 0/0 00000100:00100000:1.0:1789762979.096122:0:14170:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1189 00000100:00100000:1.0:1789762979.096136:0:14170:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452330068705 time 452330167973 state 0x2 ref 0 00000100:00100000:3.0:1789762979.098621:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762979.098680:0:13300:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067578112 00000100:00100000:2.0:1789762979.098688:0:13300:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067578112 00000100:00100000:2.0:1789762979.098729:0:13300:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452330068705 time 452330167973 state 0x0 ref 1 00000100:00100000:2.0:1789762979.098740:0:13300:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452530167973 time 452330167973 state 0x1 ref 1 00000100:00100000:1.0:1789762979.283190:0:13301:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452530167973 time 452530256113 state 0x0 ref 1 00000100:00100000:1.0:1789762979.283217:0:13301:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067578112:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762979.283268:0:13301:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 35651584, mtime: 0x6aad9da2, atime: 0x0, ctime: 0x6aad9da2, blocks: 0x11007 00010000:00010000:1.0:1789762979.283282:0:13301:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d75 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.283295:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=452s v=(0 0) 00010000:00010000:1.0:1789762979.283299:0:13301:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762979.283304:0:13301:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d75 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.283314:0:13301:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d75 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762979.295553:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968185 is committed 00002000:00010000:2.0:1789762979.295581:0:13301:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 35651584 -> 36700160 00002000:00010000:2.0:1789762979.295586:0:13301:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 69639 -> 71687 00010000:00010000:2.0:1789762979.295596:0:13301:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d75 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762979.295605:0:13301:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d75 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762979.295612:0:13301:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d75 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762979.295617:0:13301:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81aa00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:2.0:1789762979.295620:0:13301:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d75 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762979.295627:0:13301:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d75 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762979.295643:0:13301:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d75 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762979.295649:0:13301:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000030b10978/0xdf5a7ac59f177d75 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13301 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762979.295713:0:13301:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067578112:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12506us (197090us total) trans 4294968185 rc 0/0 00000100:00100000:2.0:1789762979.295722:0:13301:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1190 00000100:00100000:2.0:1789762979.295731:0:13301:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452530167973 time 452530256113 state 0x2 ref 0 00000100:00100000:3.0:1789762979.297909:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762979.297988:0:14171:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067578240 00000100:00100000:1.0:1789762979.297997:0:14171:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067578240 00000100:00100000:1.0:1789762979.298039:0:14171:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452530167973 time 452530256113 state 0x0 ref 1 00000100:00100000:1.0:1789762979.298050:0:14171:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452730256113 time 452530256113 state 0x1 ref 1 00000100:00100000:0.0:1789762979.328039:0:3425:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9c37d8cf5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074769280:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789762979.328108:0:3425:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762979.328198:0:6424:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702074769280 00000100:00100000:1.0:1789762979.328207:0:6424:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702074769280 00000100:00100000:1.0:1789762979.328248:0:6424:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:1.0:1789762979.328257:0:6424:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c37d8cf6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074769280:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789762979.328289:0:6424:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789762979.328335:0:6424:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c37d8cf6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3425:x1876702074769280:12345-0@lo:41:osp-pre-1-0.0 Request processed in 80us (227us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762979.328343:0:6424:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00100000:0.0:1789762979.328376:0:3425:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9c37d8cf5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3425:1876702074769280:0@lo:41:osp-pre-1-0.0 00010000:00010000:1.0:1789762979.456094:0:6408:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762979.483240:0:8059:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452730256113 time 452730306026 state 0x0 ref 1 00000100:00100000:1.0:1789762979.483278:0:8059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067578240:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762979.483329:0:8059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 36700160, mtime: 0x6aad9da2, atime: 0x0, ctime: 0x6aad9da2, blocks: 0x11807 00010000:00010000:1.0:1789762979.483336:0:8059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d7c lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.483350:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=452s v=(0 0) 00010000:00010000:1.0:1789762979.483354:0:8059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762979.483370:0:8059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d7c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.483381:0:8059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d7c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762979.495797:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968186 is committed 00002000:00010000:1.0:1789762979.495802:0:8059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 36700160 -> 37748736 00002000:00010000:1.0:1789762979.495807:0:8059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime from disk: 1789762978 -> 1789762979 00002000:00010000:1.0:1789762979.495810:0:8059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime from disk: 1789762978 -> 1789762979 00002000:00010000:1.0:1789762979.495812:0:8059:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 71687 -> 73735 00010000:00010000:1.0:1789762979.495821:0:8059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d7c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.495831:0:8059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d7c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.495838:0:8059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d7c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.495844:0:8059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762979.495847:0:8059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d7c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.495854:0:8059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d7c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.495867:0:8059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d7c lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.495872:0:8059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d7c lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762979.495953:0:8059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2397:x1876702067578240:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12685us (198043us total) trans 4294968186 rc 0/0 00000100:00100000:1.0:1789762979.495961:0:8059:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1191 00000100:00100000:1.0:1789762979.495971:0:8059:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452730256113 time 452730306026 state 0x2 ref 0 00000100:00100000:3.0:1789762979.498220:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:1.0:1789762979.498256:0:14170:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067578368 00000100:00100000:1.0:1789762979.498264:0:14170:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067578368 00000100:00100000:1.0:1789762979.498313:0:14170:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452730256113 time 452730306026 state 0x0 ref 1 00000100:00100000:1.0:1789762979.498327:0:14170:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452930306026 time 452730306026 state 0x1 ref 1 00000100:00100000:1.0:1789762979.683314:0:8058:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452930306026 time 452930380048 state 0x0 ref 1 00000100:00100000:1.0:1789762979.683343:0:8058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9c36d284e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067578368:12345-192.168.203.37@tcp:4:dd.500 00002000:00010000:1.0:1789762979.683396:0:8058:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 37748736, mtime: 0x6aad9da3, atime: 0x0, ctime: 0x6aad9da3, blocks: 0x12007 00010000:00010000:1.0:1789762979.683403:0:8058:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d83 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.683416:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=452s v=(0 0) 00010000:00010000:1.0:1789762979.683419:0:8058:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762979.683424:0:8058:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d83 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.683435:0:8058:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d83 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762979.696155:0:8082:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968187 is committed 00002000:00010000:1.0:1789762979.696167:0:8058:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:1.0:1789762979.696172:0:8058:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 73735 -> 75783 00010000:00010000:1.0:1789762979.696182:0:8058:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d83 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.696192:0:8058:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d83 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.696203:0:8058:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d83 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.696208:0:8058:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c37ee81be00 already unused, calling callback (ffffffffc1554350) 00010000:00010000:1.0:1789762979.696212:0:8058:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d83 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.696218:0:8058:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d83 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.696231:0:8058:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d83 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.696236:0:8058:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cbd22e9/0xdf5a7ac59f177d83 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8058 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762979.696302:0:8058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9c36d284e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:fa6fc127-0d58-47ca-82d1-2c8ad7b086c1+5:2398:x1876702067578368:12345-192.168.203.37@tcp:4:dd.500 Request processed in 12968us (198078us total) trans 4294968187 rc 0/0 00000100:00100000:1.0:1789762979.696314:0:8058:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.37@tcp, seq: 1192 00000100:00100000:1.0:1789762979.696326:0:8058:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452930306026 time 452930380048 state 0x2 ref 0 00000100:00100000:3.0:1789762979.698498:0:3417:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.37@tcp (source: 12345-192.168.203.37@tcp) 00000100:00100000:2.0:1789762979.698557:0:13301:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702067578496 00000100:00100000:2.0:1789762979.698565:0:13301:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702067578496 00000100:00100000:2.0:1789762979.698608:0:13301:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 452930306026 time 452930380048 state 0x0 ref 1 00000100:00100000:2.0:1789762979.698623:0:13301:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9c37ff997600 key jobid={dd.500}&nid={192.168.203.37@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff9c37d91d4a00 gen 0 rate 5 depth 3 token 0 deadline 453130380048 time 452930380048 state 0x1 ref 1 Debug log: 62706 lines, 62706 kept, 0 dropped, 0 bad.